1 Commits

Author SHA1 Message Date
Spencer Brower
7df83b064e chore(odin): release 0.3.0 2026-06-16 11:39:29 -04:00
52 changed files with 1939 additions and 2945 deletions

View File

@@ -2,6 +2,8 @@ on:
push:
branches:
- main
- dev
- odin
permissions:
contents: write

5
.gitignore vendored
View File

@@ -7,13 +7,8 @@ list.json
man
# build artifacts
*.spall
builds
envr
envr-go
envr-prof
findr/findr
findr/findr-prof
findr/bench-*.md
result
version.odin

View File

@@ -2,10 +2,6 @@
## [0.3.0](https://github.com/sbrow/envr/compare/v0.2.1...v0.3.0) (2026-06-16)
Version 0.3.0 represents a significant departure (and improvement) for envr.
The entire codebase was rewritten in [Odin](https://odin-lang.org/) (from Go).
This reduced the binary size from over 17MB to under 600k, improved performance,
and significantly reduced the number of project dependencies from 69 to just 2.
### ⚠ BREAKING CHANGES

View File

@@ -10,7 +10,7 @@ LINUX_AMD64_BIN := $(BUILD_DIR)/$(APP_NAME)-$(VERSION)-linux-amd64
LINUX_ARM64_BIN := $(BUILD_DIR)/$(APP_NAME)-$(VERSION)-linux-arm64
DARWIN_ARM64_BIN := $(BUILD_DIR)/$(APP_NAME)-$(VERSION)-darwin-arm64
.PHONY: all clean cleanall build-linux build-darwin compress release profile help
.PHONY: all clean cleanall build-linux build-darwin compress release help
# Default target
all: release clean
@@ -66,12 +66,6 @@ release: build-linux compress
@echo "Release artifacts created:"
@ls -la $(BUILD_DIR)/*.tar.gz $(BUILD_DIR)/*.zip 2>/dev/null || echo "No compressed artifacts found"
# Build with spall profiling instrumentation
profile:
@echo "Building with spall profiling..."
odin build . -define:SPALL=true -o:speed -out:envr-prof
@echo "Built envr-prof (run it to generate envr.spall)"
# Clean binary files only
clean:
@echo "Cleaning binary files..."
@@ -90,7 +84,6 @@ help:
@echo " build-linux - Build Linux binaries only"
@echo " build-darwin - Build Darwin binaries only"
@echo " compress - Compress all built binaries"
@echo " profile - Build with spall profiling instrumentation"
@echo " clean - Remove binary files only"
@echo " cleanall - Remove entire build directory"
@echo " help - Show this help message"

View File

@@ -12,13 +12,14 @@ the tool [of your choosing](#backup-options).
## Features
- **Encrypted Storage**: All `.env` files are encrypted using your ssh key and
[libsodium](https://github.com/jedisct1/libsodium).
- **Automatic Sync**: Update the database with one command, which can easily
- 🔐 **Encrypted Storage**: All `.env` files are encrypted using your ssh key and
[libsodium](https://github.com/jedisct1/libsodium) encryption.
- 🔄 **Automatic Sync**: Update the database with one command, which can easily
be run on a cron.
- **Smart Scanning**: Automatically discover and import `.env` files in your
- 🔍 **Smart Scanning**: Automatically discover and import `.env` files in your
home directory.
- **Rename Detection**: Automatically find and updates renamed/moved
- **Interactive CLI**: User-friendly prompts for file selection and management.
- 🗂️ **Rename Detection**: Automatically finds and updates renamed/moved
repositories.
## TODOS
@@ -27,10 +28,14 @@ repositories.
- [x] Allow configuration of ssh key.
- [x] Allow multiple ssh keys.
## Installation
## Prerequisites
You will need an SSH key pair for encryption and decryption. You can generate one
with `ssh-keygen -t ed25519`. It will be saved to `~/.ssh/id_ed25519`.
- An SSH key pair (for encryption/decryption)
- The following binaries:
- [fd](https://github.com/sharkdp/fd)
- [git](https://git-scm.com)
## Installation
### With Odin
@@ -91,12 +96,7 @@ The configuration file is created during initialization:
],
"scan": {
"matcher": "\\.env",
"exclude": [
"*\\.envrc",
"\\.local/",
"node_modules",
"vendor"
],
"exclude": "*.envrc",
"include": "~"
}
}

268
TABLE_IMPROVEMENT_PLAN.md Normal file
View File

@@ -0,0 +1,268 @@
# Table Rendering Memory Optimization Plan
## Executive Summary
This plan outlines improvements to eliminate excessive memory allocations and copies in the Odin table rendering system. The current implementation makes 10+ allocations per row, while the Zig equivalent makes zero allocations for rendering. This optimization will reduce memory usage, improve performance, and align with the project's efficiency goals.
## Current State Analysis
### Zig Version (Reference Implementation)
- **Allocations**: 1 (data only)
- **Data copies**: 0
- **String allocation**: 0
- **Column widths**: Stack array
- **Output**: Direct to writer
### Odin Version (Current Implementation)
- **Allocations**: 10+ per row
- **Data copies**: Multiple per row
- **String allocation**: 2+ per row (concatenate + slice)
- **Column widths**: Heap allocated
- **Output**: Builder → stdout
### Current Issues Identified
1. **Table Infrastructure** (`table.odin`)
- Uses `strings.Builder` which allocates per-line memory
- Heap-allocated `[dynamic]int` for column widths
- Multiple `strings.concatenate()` calls creating new strings
2. **Command Implementations**
- `cmd_list`: Creates intermediate `[]string` slices per row, allocates new strings via `strings.concatenate()`
- `cmd_sync`: Creates `SyncEntry` structs with cloned strings, allocates dynamic arrays
- `cmd_deps`: Allocates dynamic rows array unnecessarily
3. **Memory Pattern**
- Each command allocates `[][]string` for table data
- Manual struct-to-row transformation creates copies
- Duplicate code across all table-using commands
## Proposed Solutions
### Phase 1: Core Table Infrastructure Overhaul
#### 1.1 Direct Writer-Based Rendering
**Current:**
```odin
b: strings.Builder
strings.builder_init(&b)
// ... build table in builder
fmt.println(strings.to_string(b))
```
**Proposed:**
```odin
render_table :: proc(writer: io.Writer, headers: []string, rows: [][]string)
```
- Replace `strings.Builder` with `io.Writer` output
- Eliminate intermediate string allocations
- Write table components directly to output stream
#### 1.2 Stack-Based Column Widths
**Current:**
```odin
col_widths := make([dynamic]int, 0, len(headers))
```
**Proposed:**
- Use fixed stack arrays for reasonable column counts
- Implement small buffer optimization (SBO) for variable column counts
- Only allocate for tables exceeding threshold (e.g., 16 columns)
#### 1.3 Zero-Copy String Handling
**Current:**
```odin
dir_str := strings.concatenate({row.Dir, "/"}, context.temp_allocator)
```
**Proposed:**
- Replace `strings.concatenate()` with string slicing
- Work directly with `EnvFile.Path` and `EnvFile.Dir` fields
- Use `filepath.base()` and `filepath.dir()` without allocation where possible
### Phase 2: Generic Table Interface
#### 2.1 Field-Based Table Renderer
```odin
Table_Field :: struct {
name: string,
value: string, // String view, no allocation
alignment: Alignment,
}
Table_Config :: struct {
writer: io.Writer,
fields: []Table_Field,
col_widths: []int,
}
render_row :: proc(cfg: Table_Config, row_data: any)
```
- Accept struct fields directly without intermediate arrays
- Support field selection (show only specific fields)
- Alignment options (left/center/right)
#### 2.2 Field Extraction Procs
- Generate field extraction helpers for each struct type
- Avoid string allocation by returning string views
- Cache computed values (like formatted status strings)
#### 2.3 Streaming Table Processing
- Process rows one at a time without collecting all rows
- Reduce peak memory usage from O(N × strings) to O(table_structure)
- Enable early termination if needed
### Phase 3: Command-Specific Optimizations
#### 3.1 Eliminate Intermediate Structs
**Current (cmd_sync):**
```odin
for &file in files {
// ... processing
path_str, _ := strings.clone(file.Path)
status_str, _ := strings.clone(status)
append(&results, SyncEntry{Path = path_str, Status = status_str})
}
```
**Proposed:**
```odin
for &file in files {
result, err_msg := db_sync(&db, &file)
// Direct rendering with zero-copy
render_sync_row(writer, file, result, err_msg)
}
```
- `cmd_sync`: Work directly with `EnvFile` + `SyncFlagEnum`
- `cmd_list`: Use `EnvFile` fields directly, no `ListEntry`
- Generate table content on-the-fly
#### 3.2 In-Place Status Computation
```odin
get_sync_status :: proc(result: SyncFlag, err_msg: string) -> string {
switch {
case .Error in result: return if len(err_msg) > 0 then err_msg else "error"
case .BackedUp in result: return "Backed Up"
case .Restored in result: return "Restored"
case .DirUpdated in result: return "Moved"
case: return "OK"
}
}
```
- Compute status strings without allocation (use static lookup)
- Cache formatted status values if needed
- Reduce allocation count from N to 0 or 1
#### 3.3 Batch Processing
- Reduce allocation count by pooling small allocations
- Use `context.temp_allocator` more effectively
- Pre-allocate buffers for expected sizes
### Phase 4: JSON Output Separation
#### 4.1 Unified JSON Rendering
```odin
render_json_rows :: proc(writer: io.Writer, rows: any, field_names: []string)
```
- Create centralized JSON rendering helper
- Work with same structs as table rendering
- Use reflection or explicit field marshaling
#### 4.2 Format-Agnostic Interface
- Commands generate data → renderers handle format
- Table renderer focuses only on ASCII/Unicode output
- Keep terminal detection in command layer
## Expected Improvements
| Metric | Current | Target | Improvement |
|--------|---------|--------|-------------|
| **Allocations** | 10+ per row | 0-1 per table | 10x+ reduction |
| **Memory copies** | 2-3 per row | 0 | 100% reduction |
| **Peak memory** | O(N × strings) | O(table_structure) | Constant factor |
| **Throughput** | Baseline | 2-3x faster | Performance boost |
## Implementation Strategy
### High-Priority Changes
1. Replace `strings.Builder` with direct `io.Writer` output
2. Convert column widths to stack-based allocation
3. Eliminate intermediate struct allocations in commands
### Medium-Priority Changes
1. Create generic field-based table interface
2. Implement streaming table processing
3. Centralize JSON rendering logic
### Low-Priority Changes
1. Add alignment options beyond left-aligned
2. Implement comprehensive field introspection
3. Add advanced table formatting features
## Tradeoff Questions
Before implementation begins, we need to resolve these architectural questions:
### 1. Generality vs. Performance
**Question:** Should we create a fully generic table renderer (similar to Zig's `Table(T)`) or focus on optimizing the current 3 use cases first?
**Options:**
- **Generic approach**: Higher development cost, future-proof, may have some overhead
- **Specific optimization**: Faster implementation, maximum performance for current use cases, less flexible
**Recommendation:** Start with specific optimizations for current use cases, then generalize patterns that emerge.
### 2. Alignment Support
**Question:** Does the project need left/center/right alignment support, or is left-alignment sufficient?
**Context:** Zig supports alignment options, but current Odin implementation only left-aligns. Most CLI tables work fine with left alignment.
**Recommendation:** Start with left-alignment only, add alignment if specific use cases demand it.
### 3. API Compatibility
**Question:** Should we maintain the current `render_table()` API signature, or are breaking changes acceptable?
**Current API:**
```odin
render_table :: proc(headers: []string, rows: [][]string)
```
**Options:**
- **Maintain API**: Slower to implement, backward compatible, may need adapter layers
- **Break API**: Faster implementation, cleaner code, requires updates to all callers
**Recommendation:** Breaking changes are acceptable since this is an optimization-focused effort and callers are limited to 3 commands.
### 4. Odin Capabilities
**Question:** What runtime reflection or field introspection capabilities does Odin provide?
**Context:** Zig uses `@typeInfo()` and comptime field iteration. We need to understand Odin's equivalent capabilities to design the optimal solution.
**Recommendation:** Investigate Odin's runtime type information capabilities before finalizing the generic table interface design.
### 5. Testing Strategy
**Question:** Should we add comprehensive tests for new table rendering before optimizing commands, or optimize incrementally with tests added afterwards?
**Options:**
- **Test-first**: More robust, catches regressions early, slower initial development
- **Optimize-first**: Faster development, may miss edge cases, requires retroactive testing
**Recommendation:** Hybrid approach - add basic tests for core infrastructure, then optimize incrementally with additional tests for each command.
## Next Steps
1. **Research Phase**: Investigate Odin's type system and reflection capabilities
2. **Prototype Phase**: Create minimal working prototype of zero-allocation table renderer
3. **Refactor Phase**: Incrementally update commands to use new infrastructure
4. **Test Phase**: Add comprehensive tests and verify memory improvements
5. **Benchmark Phase**: Measure performance improvements and memory usage
## Success Criteria
- [ ] Zero allocations for table rendering (excluding initial data)
- [ ] Zero string copies in the happy path
- [ ] All 3 commands (`list`, `sync`, `deps`) use new infrastructure
- [ ] Performance improvement of 2x or more
- [ ] Memory usage reduction of 50% or more
- [ ] No regression in table formatting quality
- [ ] Backward compatibility with JSON output format

View File

@@ -35,7 +35,13 @@ Stdout will be captured by redirecting `os.stdout` to a pipe.
## Hard to test (interactive / external deps)
### `cmd_deps` (cmd_deps.odin)
- Needs `git` and/or `fd` in PATH
- Test TTY and non-TTY paths
- Skip if dependencies not available (with `#assert` like TODO 28 suggests)
### `cmd_scan` (cmd_scan.odin)
- Needs `fd` installed
- Test with fixture git repo containing `.env` files
- Test `find_unbacked` integration (already partially tested in `cmd_check_test.odin`)
- Non-TTY JSON output path
@@ -60,4 +66,5 @@ Stdout will be captured by redirecting `os.stdout` to a pipe.
- DB integration tests should use in-memory SQLite (`:memory:`) where possible.
- Temp dir fixtures should follow the pattern in `scan_test.odin`.
- External dependency tests (`fd`, `git`) should use `#assert` to ensure the dependency is present rather than silently skipping (TODO 28).
- Tests that manipulate the `HOME` env var must use a mutex to prevent races with parallel test execution.

View File

@@ -1,18 +1,13 @@
# TODOs
1. Commands are still leaking.
28. **db.odin** — Inconsistencies in how struct vs sqlite are named.
1. Consider giving db its own allocator
29. Add color flag and support non colored output.
2. **db.odin:324-327** — Map iteration (`remote_set`) is non-deterministic. Same file can produce different JSON on each backup, causing spurious DB diffs. Sort remotes before storing.
30. Use text/tables for command output
3. **db.odin:135, 250** — String interpolation into SQL (`VACUUM INTO '%s'`, `ATTACH DATABASE '%s'`). Currently safe because input is controlled, but fragile.
2. Generate md and man pages again.
3. **db.odin:324-327** — Map iteration (`remote_set`) is non-deterministic. Same file can produce different JSON on each backup, causing spurious DB diffs. Sort remotes before storing.
4. Make sure official path separators are used when appropriate, rather than '/'.
4. **features.odin:30-41**`find_binary` uses `strings.join` instead of `filepath.join`, uses `os.stat` instead of checking executability, hardcodes `:` as PATH separator (wrong on Windows).
5. **cmd_restore.odin:20-30 & cmd_remove.odin:19-29** — Identical path-resolution block copy-pasted. `is_abs` guard is redundant since `filepath.abs` is a no-op on absolute paths. Extract a helper.
@@ -24,42 +19,38 @@
11. **db.odin:352-353**`hex.encode` error ignored. `string(hex_bytes)` aliases the byte slice.
12. **cmd_sync.odin:80, cmd_list.odin:33**`make([]string, 2)` for table rows never freed. Leaks per row. Defer to memory pass.
12. **cmd_sync.odin:80, cmd_list.odin:33, cmd_deps.odin:9**`make([]string, 2)` for table rows never freed. Leaks per row. Defer to memory pass.
13. Check for prealloc opportunities. i.e. `make([dynamic]string)` -> `make([dynamic]string, 5)`.
13. **cmd_list.odin** — Non-TTY branch builds `ListEntry` structs and marshals JSON separately. Now that `render_json_rows` (issue 1) accepts an `io.Writer` and uses `json.marshal`, unify both branches to use it. Note: will change JSON keys from `"directory"/"path"` to `"Directory"/"Path"`.
14. Add a text filter to the multi_select.
14. Check for prealloc opportunities. i.e. `make([dynamic]string)` -> `make([dynamic]string, 5)`.
16. Add tests for untested commands.
15. Add a text filter to the multi_select.
17. 2 scan tests silently skip when fd isn't installed, tests pass without actually testing anything. These should use #assert to be sure that fd is in path.
16. Create backup / fallback fd.
19. add --format -f flag to commands that draw tables.
17. Add tests for untested commands.
20. Replace `testing.expect` calls with `testing.expect_value` calls where appropriate.
18. 2 scan tests silently skip when fd isn't installed, tests pass without actually testing anything. These should use #assert to be sure that fd is in path.
21. Change struct field names from PascalCase to snake_case.
19. Try to do all encryption / decryption in memory - only read / write encrypted data to disk.
20. add --format -f flag to commands that draw tables.
21. Replace `testing.expect` calls with `testing.expect_value` calls where appropriate.
22. Change struct field names from PascalCase to snake_case.
23. procedures should be ordered by use, main at the top, then in the order they are called from main.
24. Shell completion
25. Bring back windows support / cross-compilation.
26. Test all cmds / terminal branches.
27. Replace `fmt.tprintf("/tmp/envr-test-...-%d", os.get_pid())` + `os.mkdir_all` in test files with `os.mkdir_temp` (race-free, honors `$TMPDIR`, matches `findr/test_env.odin` pattern).
28. Adopt `core:log` across `db.odin`, `crypto.odin`, `config.odin`, `ssh.odin` — replace ~30 scattered `fmt.printf("Error ...")` calls with leveled logging for consistent stderr routing and source locations.
## Double-check AI output
- [ ] cli.odin
- [ ] cli_test.odin
- [x] colors.odin
- [x] cmd_backup.odin
- [x] cmd_check.odin
- [ ] cmd_check_test.odin
- [x] cmd_deps.odin
- [ ] cmd_edit_config.odin
- [x] cmd_init.odin
- [x] cmd_list.odin
@@ -78,6 +69,8 @@
- [ ] db.odin
- [ ] db_integration_test.odin
- [ ] db_test.odin
- [x] features.odin
- [x] features_test.odin
- [x] main.odin
- [x] prompt.odin
- [ ] scan.odin
@@ -88,11 +81,3 @@
- [ ] ssh_test.odin
- [ ] table.odin
- [ ] table_test.odin
- [ ] findr/findr_test.odin
- [ ] findr/gitignore.odin
- [ ] findr/gitignore_test.odin
- [ ] findr/glob.odin
- [ ] findr/glob_test.odin
- [ ] findr/repos.odin
- [ ] findr/test_env.odin
- [ ] findr/walker.odin

92
WINDOWS.md Normal file
View File

@@ -0,0 +1,92 @@
# Windows Compatibility Guide
This document outlines Windows compatibility issues and solutions for the envr project.
## Critical Issues
### 1. Path Handling Bug (MUST FIX)
**File:** `app/env_file.go:209`
**Issue:** Uses `path.Join` instead of `filepath.Join`, which won't work correctly on Windows due to different path separators.
**Current code:**
```go
f.Path = path.Join(newDir, path.Base(f.Path))
```
**Fixed code:**
```go
f.Path = filepath.Join(newDir, filepath.Base(f.Path))
```
## External Dependencies
The application relies on external tools that need to be installed separately on Windows:
### Required Tools
1. **fd** - Fast file finder
- Install via: `winget install sharkdp.fd` or `choco install fd`
- Alternative: `scoop install fd`
2. **git** - Version control system
- Install via: `winget install Git.Git` or download from git-scm.com
- Usually already available on most development machines
## Minor Compatibility Notes
### File Permissions
- Unix file permissions (`0755`, `0644`) are used throughout the codebase
- These are safely ignored on Windows - no changes needed
### Editor Configuration
**File:** `cmd/edit_config.go:20-24`
**Issue:** Relies on `$EDITOR` environment variable which is less common on Windows.
**Current behavior:** Fails if `$EDITOR` is not set
**Recommended improvement:** Add fallback detection for Windows editors:
```go
editor := os.Getenv("EDITOR")
if editor == "" {
if runtime.GOOS == "windows" {
editor = "notepad.exe" // or "code.exe" for VS Code
} else {
fmt.Println("Error: $EDITOR environment variable is not set")
return
}
}
```
## Installation Instructions for Windows
1. Install required dependencies:
```powershell
winget install sharkdp.fd
winget install Git.Git
```
2. Fix the path handling bug in `app/env_file.go:209`
3. Build and run as normal:
```powershell
go build
.\envr.exe init
```
## Testing on Windows
After applying the critical path fix, the core functionality should work correctly on Windows. The application has been designed with cross-platform compatibility in mind, using:
- `filepath` package for path operations (mostly)
- `os.UserHomeDir()` for home directory detection
- Standard Go file operations
## Summary
- **1 critical bug** must be fixed for Windows compatibility
- **2 external tools** need to be installed
- **1 minor enhancement** recommended for better Windows UX
- Overall architecture is Windows-compatible

196
cli.odin
View File

@@ -43,6 +43,13 @@ key somewhere, otherwise your data could be lost forever.`,
{"list", "envr list", "View your tracked files", "", {}},
{"remove", "envr remove <path>", "Remove a .env file from your database", "", {}},
{"check", "envr check [path]", "Check if files are backed up", "", {}},
{
"deps",
"envr deps",
"Check for missing binaries",
"envr relies on external binaries for certain functionality.\n\nThe check command reports on which binaries are available and which are not.",
{},
},
{"version", "envr version", "Show envr's version", "", {}},
{"edit-config", "envr edit-config", "Edit your config with your default editor", "", {}},
{
@@ -54,12 +61,20 @@ key somewhere, otherwise your data could be lost forever.`,
},
}
delete_command :: proc(cmd: ^Command) {
delete(cmd.args)
delete(cmd.flags)
delete(cmd.bool_set)
bufio.writer_destroy(cmd.out_buf)
free(cmd.out_buf)
}
// Caller is responsible for calling delete_command(cmd).
// FIXME: Works in kinda a wonky and awkward way.
parse_args :: proc(args: []string, out: io.Stream, err: io.Stream) -> (cmd: Command, ok: bool) {
{
cmd.out_buf = new(bufio.Writer)
bufio.writer_init(cmd.out_buf, out, allocator = context.allocator)
bufio.writer_init(cmd.out_buf, out)
cmd.out = bufio.writer_to_writer(cmd.out_buf)
cmd.err = err
}
@@ -75,7 +90,6 @@ parse_args :: proc(args: []string, out: io.Stream, err: io.Stream) -> (cmd: Comm
cmd.flags = make(map[string]string)
cmd.bool_set = make(map[string]bool)
// TODO: Optimize loop?
i := 2
for i < len(args) {
arg := args[i]
@@ -103,10 +117,9 @@ parse_args :: proc(args: []string, out: io.Stream, err: io.Stream) -> (cmd: Comm
}
}
val: string = ---
if val, ok = cmd.flags["config-file"]; ok {
if val, ok := cmd.flags["config-file"]; ok {
cmd.config_path = val
} else if val, ok = cmd.flags["c"]; ok {
} else if val, ok := cmd.flags["c"]; ok {
cmd.config_path = val
} else {
// FIXME: Handle err
@@ -124,80 +137,13 @@ parse_args :: proc(args: []string, out: io.Stream, err: io.Stream) -> (cmd: Comm
return cmd, true
}
print_command_help :: proc(cmd: ^Command) {
ok := write_command_help(cmd.name, cmd.out)
if !ok {
fmt.wprintf(cmd.err, "Unknown command: %s\n", cmd.name)
write_usage(cmd.out)
}
}
write_command_help :: proc(name: string, w: io.Writer) -> bool {
info, found := find_command(name)
if !found {
return false
}
fmt.wprintf(
w,
"%s\n\n\n" +
COLOR_HEADINGS +
"Usage:" +
ANSI_RESET +
"\n\n " +
COLOR_FLAGS +
"%s" +
ANSI_RESET +
" [flags]\n\n",
info.short,
info.usage,
flush = false,
)
if len(info.aliases) > 0 {
fmt.wprintf(
w,
"\n" +
COLOR_HEADINGS +
"Aliases:" +
ANSI_RESET +
"\n\n " +
COLOR_COMMANDS +
"%s" +
ANSI_RESET,
info.name,
flush = false,
)
for a in info.aliases {
fmt.wprintf(w, ", " + COLOR_COMMANDS + "%s" + ANSI_RESET, a, flush = false)
}
fmt.wprintf(w, "\n", flush = false)
}
if len(info.long) > 0 {
fmt.wprintf(w, "\n%s\n", info.long, flush = false)
}
fmt.wprintf(
w,
"\n" +
COLOR_HEADINGS +
"Flags:" +
ANSI_RESET +
"\n\n " +
COLOR_FLAGS +
"-h, --help" +
ANSI_RESET +
" help for %s\n " +
COLOR_FLAGS +
"-c, --config-file" +
ANSI_RESET +
` <path> config file (default "~/.envr/config.json")
`,
info.name,
flush = false,
)
has_flag :: proc(cmd: ^Command, name: string) -> bool {
_, ok := cmd.flags[name]
if ok {
return true
}
_, ok2 := cmd.bool_set[name]
return ok2
}
find_command :: proc(name: string) -> (CommandInfo, bool) {
@@ -214,15 +160,53 @@ find_command :: proc(name: string) -> (CommandInfo, bool) {
return CommandInfo{}, false
}
write_command_help :: proc(name: string, w: io.Writer) -> bool {
info, found := find_command(name)
if !found {
return false
}
fmt.wprintf(w, "Usage: %s [flags]\n\n", info.usage, flush = false)
fmt.wprintf(w, "%s\n", info.short, flush = false)
if len(info.aliases) > 0 {
fmt.wprintf(w, "\nAliases:\n %s", info.name, flush = false)
for a in info.aliases {
fmt.wprintf(w, ", %s", a, flush = false)
}
fmt.wprintf(w, "\n", flush = false)
}
if len(info.long) > 0 {
fmt.wprintf(w, "\n%s\n", info.long, flush = false)
}
fmt.wprintf(
w,
"\nFlags:\n -h, --help help for %s\n -c, --config-file <path> config file (default \"~/.envr/config.json\")\n",
info.name,
flush = false,
)
return true
}
print_command_help :: proc(cmd: ^Command) {
ok := write_command_help(cmd.name, cmd.out)
if !ok {
fmt.wprintf(cmd.err, "Unknown command: %s\n", cmd.name)
write_usage(cmd.out)
}
}
// TODO: command args should be shown in usage.
write_usage :: proc(w: io.Writer) {
fmt.wprintf(
w,
`envr keeps your .env synced to a local, encrypted database.
`envr keeps your .env synced to a local, age encrypted database.
Is a safe and easy way to gather all your .env files in one place where they can
easily be backed by another tool such as restic or git.
All your data is stored in ~/.envr/data.envr
All your data is stored in ~/data.age
Getting started is easy:
@@ -249,29 +233,21 @@ at before, restore your backup with:
> envr restore ~/<path to repository>/.env
%sUsage:%s
Usage:
envr [command]
%senvr%s [command]
%sAvailable Commands:%s
Available Commands:
`,
COLOR_HEADINGS,
ANSI_RESET,
COLOR_FLAGS,
ANSI_RESET,
COLOR_HEADINGS,
ANSI_RESET,
flush = false,
)
for c in COMMANDS {
name_start := len(c.name)
fmt.wprintf(w, " %s%s", COLOR_COMMANDS, c.name, flush = false)
fmt.wprintf(w, "%s", c.name, flush = false)
for a in c.aliases {
fmt.wprintf(w, ", %s", a, flush = false)
name_start += len(a) + 2
}
fmt.wprint(w, ANSI_RESET)
padding := 20 - name_start
if padding > 0 {
for _ in 0 ..< padding {
@@ -283,40 +259,14 @@ at before, restore your backup with:
fmt.wprintf(
w,
"\n" +
COLOR_HEADINGS +
"Flags:" +
ANSI_RESET +
"\n\n " +
COLOR_FLAGS +
"-h, --help" +
ANSI_RESET +
" help for envr\n" +
COLOR_FLAGS +
` -c, --config-file` +
ANSI_RESET +
` <path> config file (default "~/.envr/config.json")
`
Flags:
-h, --help help for envr
-c, --config-file <path> config file (default "~/.envr/config.json")
Use "` +
COLOR_FLAGS +
"envr" +
ANSI_RESET +
` [command] --help" for more information about a command.
Use "envr [command] --help" for more information about a command.
`,
flush = false,
)
}
has_flag :: proc(cmd: ^Command, name: string) -> bool {
return name in cmd.flags || name in cmd.bool_set
}
delete_command :: proc(cmd: ^Command) {
bufio.writer_flush(cmd.out_buf)
delete(cmd.args)
delete(cmd.flags)
delete(cmd.bool_set)
bufio.writer_destroy(cmd.out_buf)
free(cmd.out_buf)
}

View File

@@ -1,6 +1,5 @@
#+feature dynamic-literals
package main
package main
import "core:bufio"
import "core:fmt"
@@ -58,7 +57,7 @@ test_usage_text_contains_flags_and_help_hint :: proc(t: ^testing.T) {
testing.expect(t, strings.contains(text, "Flags:"), "missing Flags section")
testing.expect(t, strings.contains(text, "--help"), "missing --help flag")
testing.expect(t, strings.contains(text, "Use \"envr [command] --help\""), "missing help hint")
}
}
@(test)
test_command_help_backup :: proc(t: ^testing.T) {

View File

@@ -4,9 +4,9 @@ import "core:fmt"
import "core:os"
import "core:path/filepath"
// TODO: What happens if you pass a non existent path to cmd_check?
// TODO: UX could be improved, so "run envr add ." if file not exists.
cmd_check :: proc(cmd: ^Command) {
feats := check_features()
check_path: string
if len(cmd.args) > 0 {
check_path = cmd.args[0]
@@ -39,10 +39,18 @@ cmd_check :: proc(cmd: ^Command) {
is_dir := os.is_directory(abs_path)
// TODO: set a reasonable default
files_in_path := make([dynamic]string, context.temp_allocator)
files_in_path: [dynamic]string
if is_dir {
if cant_scan(feats) {
fmt.wprintln(
cmd.err,
"Error: please install fd to use the check command (https://github.com/sharkdp/fd)",
flush = false,
)
return
}
scanned, scan_ok := scan_path(abs_path, db.cfg)
if !scan_ok {
fmt.wprintln(cmd.err, "Error scanning directory for .env files", flush = false)
@@ -64,23 +72,13 @@ cmd_check :: proc(cmd: ^Command) {
if len(files_in_path) == 0 {
fmt.wprintln(cmd.out, "No .env files found in the specified directory.", flush = false)
} else {
fmt.wprintln(
cmd.out,
"✓ All .env files in the directory are backed up.",
flush = false,
)
fmt.wprintln(cmd.out, "✓ All .env files in the directory are backed up.", flush = false)
}
} else {
fmt.wprintf(
cmd.out,
"Found %d .env file(s) that are not backed up:\n",
len(not_backed),
flush = false,
)
fmt.wprintf(cmd.out, "Found %d .env file(s) that are not backed up:\n", len(not_backed), flush = false)
for file in not_backed {
fmt.wprintf(cmd.out, " %s\n", file, flush = false)
}
fmt.wprintln(cmd.out, "\nRun 'envr sync' to back up these files.", flush = false)
}
}

View File

@@ -1,4 +1,3 @@
#+test
package main
import "core:fmt"

33
cmd_deps.odin Normal file
View File

@@ -0,0 +1,33 @@
package main
import "core:fmt"
import "core:os"
import "core:terminal"
// TODO: Improve table rendering
cmd_deps :: proc(cmd: ^Command) {
feats := check_features()
headers := []string{"Feature", "Status"}
rows: [dynamic][]string
if .Git in feats {
append(&rows, []string{"Git", "\u2713 Available"})
} else {
append(&rows, []string{"Git", "\u2717 Missing"})
}
if .Fd in feats {
append(&rows, []string{"fd", "\u2713 Available"})
} else {
append(&rows, []string{"fd", "\u2717 Missing"})
}
if terminal.is_terminal(os.stdout) {
render_table(cmd.out, headers, rows[:])
} else {
render_json_rows(cmd.out, headers, rows[:])
fmt.wprint(cmd.out, "\n", flush = false)
}
}

View File

@@ -12,7 +12,8 @@ cmd_edit_config :: proc(cmd: ^Command) {
config_path := cmd.config_path
if !os.exists(config_path) {
_, stat_err := os.stat(config_path, context.allocator)
if stat_err != nil {
fmt.wprintf(
cmd.err,
"Config file does not exist at %s. Run 'envr init' first.\n",

View File

@@ -1,7 +1,6 @@
package main
import "core:fmt"
import "core:terminal/ansi"
cmd_init :: proc(cmd: ^Command) {
force := has_flag(cmd, "force") || has_flag(cmd, "f")
@@ -33,7 +32,7 @@ Generate one with: ssh-keygen -t ed25519`, flush = false)
selected, result := multi_select("Select SSH private keys:", keys[:])
defer delete(selected)
if result == .Cancel {
fmt.wprintln(cmd.out, ansi.CSI + ansi.FAINT + ansi.SGR + "Cancelled." + ANSI_RESET, flush = false)
fmt.wprintln(cmd.out, "\x1b[2mCancelled.\x1b[0m", flush = false)
return
}

View File

@@ -6,7 +6,6 @@ import "core:os"
import "core:path/filepath"
import "core:strings"
import "core:terminal"
import "core:text/table"
ListEntry :: struct {
Directory: string `json:"directory"`,
@@ -26,29 +25,22 @@ cmd_list :: proc(cmd: ^Command) {
if !list_ok {
return
}
defer delete(rows)
if terminal.is_terminal(os.stdout) {
t: table.Table
table.init(&t, context.temp_allocator, context.temp_allocator)
table.padding(&t, 1, 1)
table.aligned_header_of_values(
&t,
.Center,
COLOR_TABLE_HEADING + "Directory" + ANSI_RESET,
COLOR_TABLE_HEADING + "Path" + ANSI_RESET,
)
headers := []string{"Directory", "Path"}
table_rows := make([dynamic][]string, 0, len(rows), context.temp_allocator)
for row in rows {
dir_str := strings.concatenate(
{row.Dir, os.Path_Separator_String},
context.temp_allocator,
)
dir_str := strings.concatenate({row.Dir, "/"}, context.temp_allocator)
filename := filepath.base(row.Path)
table.row(&t, dir_str, filename)
row_slice := make([]string, 2)
row_slice[0] = dir_str
row_slice[1] = filename
append(&table_rows, row_slice)
}
table.write_decorated_table(cmd.out, &t, decorations, ansi_aware_width)
render_table(cmd.out, headers, table_rows[:])
} else {
// TODO: Should we instead print full entries here?
entries: [dynamic]ListEntry

View File

@@ -1,4 +1,3 @@
#+test
package main
import "core:path/filepath"

View File

@@ -1,4 +1,3 @@
#+test
package main
import "core:fmt"

View File

@@ -4,16 +4,25 @@ import "core:encoding/json"
import "core:fmt"
import "core:os"
import "core:terminal"
import "core:terminal/ansi"
cmd_scan :: proc(cmd: ^Command) {
feats := check_features()
if cant_scan(feats) {
fmt.wprintln(
cmd.err,
"Error: please install fd to use the scan command (https://github.com/sharkdp/fd)",
flush = false,
)
return
}
db, db_ok := db_open(cmd.config_path)
if !db_ok {
return
}
defer db_close(&db)
search_dirs := search_paths(db.cfg, context.temp_allocator)
search_dirs := search_paths(db.cfg)
if len(search_dirs) == 0 {
fmt.wprintln(
cmd.err,
@@ -24,15 +33,9 @@ cmd_scan :: proc(cmd: ^Command) {
}
// TODO: Figure out a sane default
// Can't use temp allocator becuase strings inside are copied to context.allocator
all_files := make([dynamic]string)
defer {
for &f in all_files {delete(f)}
delete(all_files)
}
all_files: [dynamic]string
for dir in search_dirs {
found, scan_ok := scan_path(dir, db.cfg)
defer delete(found)
if !scan_ok {
fmt.wprintf(cmd.err, "Error scanning %s\n", dir, flush = false)
continue
@@ -72,7 +75,7 @@ cmd_scan :: proc(cmd: ^Command) {
selected, result := multi_select("Select .env files to backup:", files[:])
defer delete(selected)
if result == .Cancel {
fmt.wprintln(cmd.out, ansi.CSI + ansi.FAINT + ansi.SGR + "Cancelled." + ANSI_RESET, flush = false)
fmt.wprintln(cmd.out, "\x1b[2mCancelled.\x1b[0m", flush = false)
return
}
@@ -96,12 +99,12 @@ cmd_scan :: proc(cmd: ^Command) {
if added_count > 0 {
fmt.wprintf(
cmd.out,
ansi.CSI + ansi.BOLD + ";" + ansi.FG_GREEN + ansi.SGR + "Successfully added %d file(s) to backup." + ANSI_RESET + "\n",
"\x1b[1;32mSuccessfully added %d file(s) to backup.\x1b[0m\n",
added_count,
flush = false,
)
} else {
fmt.wprintln(cmd.out, ansi.CSI + ansi.FAINT + ansi.SGR + "No files were added." + ANSI_RESET, flush = false)
fmt.wprintln(cmd.out, "\x1b[2mNo files were added.\x1b[0m", flush = false)
}
}

View File

@@ -3,8 +3,8 @@ package main
import "core:encoding/json"
import "core:fmt"
import "core:os"
import "core:strings"
import "core:terminal"
import "core:text/table"
SyncEntry :: struct {
Path: string `json:"path"`,
@@ -24,50 +24,68 @@ cmd_sync :: proc(cmd: ^Command) {
if !list_ok {
return
}
defer delete(files)
results := make([]SyncEntry, len(files), context.temp_allocator)
// TODO: Set sane default size
results: [dynamic]SyncEntry
defer delete(results)
for &file, i in files {
result, err := db_sync(&db, &file)
for &file in files {
old_path: string
old_path, _ = strings.clone(file.Path, context.temp_allocator)
result, err_msg := db_sync(&db, &file)
status: string
if err != .None {
status = sync_error_message(err)
} else if .BackedUp in result {
status = .DirUpdated in result ? "Moved & Backed Up" : "Backed Up"
} else if .Restored in result {
status = .DirUpdated in result ? "Moved & Restored" : "Restored"
} else if .DirUpdated in result {
status = "Moved"
is_dir_updated := .DirUpdated in result
switch {
case .Error in result:
if len(err_msg) > 0 {
status = err_msg
} else {
status = "error"
}
case .BackedUp in result:
status = "Backed Up"
case .Restored in result:
status = "Restored"
case .DirUpdated in result:
status = "Moved"
case:
status = "OK"
}
results[i] = SyncEntry {
Path = file.Path,
Status = status,
if is_dir_updated {
if !db_delete(&db, old_path) {
return
}
}
if db_update_required(result) {
if !db_insert(&db, file) {
return
}
}
path_str, _ := strings.clone(file.Path)
status_str, _ := strings.clone(status)
append(&results, SyncEntry{Path = path_str, Status = status_str})
}
if terminal.is_terminal(os.stdout) {
t: table.Table
table.init(&t, context.temp_allocator, context.temp_allocator)
table.padding(&t, 1, 1)
table.aligned_header_of_values(
&t,
.Center,
COLOR_TABLE_HEADING + "File" + ANSI_RESET,
COLOR_TABLE_HEADING + "Status" + ANSI_RESET,
)
headers := []string{"File", "Status"}
table_rows := make([dynamic][]string, 0, len(results))
for res in results {
table.row(&t, res.Path, res.Status)
row_slice := make([]string, 2)
row_slice[0] = res.Path
row_slice[1] = res.Status
append(&table_rows, row_slice)
}
table.write_decorated_table(cmd.out, &t, decorations, ansi_aware_width)
render_table(cmd.out, headers, table_rows[:])
} else {
data, marshal_err := json.marshal(results[:], allocator = context.temp_allocator)
data, marshal_err := json.marshal(results[:])
if marshal_err != nil {
fmt.wprintf(cmd.err, "Error marshaling JSON: %v\n", marshal_err, flush = false)
return
@@ -76,23 +94,3 @@ cmd_sync :: proc(cmd: ^Command) {
}
}
sync_error_message :: proc(e: SyncError) -> string {
switch e {
case .None:
return ""
case .DirMissing:
return "directory missing"
case .MultipleDirs:
return "multiple directories found"
case .GitRootFailed:
return "failed to find git roots"
case .WriteFailed:
return "failed to write file"
case .ReadFailed:
return "failed to read file"
case .DbFailed:
return "failed to update database"
}
return "unknown error"
}

View File

@@ -1,17 +0,0 @@
package main
import "core:terminal/ansi"
COLOR_HEADINGS ::
ansi.CSI + ansi.FG_BRIGHT_GREEN + ";" + ansi.BOLD + ";" + ansi.UNDERLINE + ansi.SGR
COLOR_COMMANDS :: ansi.CSI + ansi.FG_BRIGHT_CYAN + ";" + ansi.BOLD + ansi.SGR
COLOR_EXAMPLE :: ansi.CSI + ansi.ITALIC + ansi.SGR
COLOR_FLAGS :: ansi.CSI + ansi.BOLD + ";" + ansi.FG_BRIGHT_WHITE + ansi.SGR
COLOR_TABLE_HEADING :: ansi.CSI + ansi.FG_BRIGHT_GREEN + ansi.SGR
ANSI_RESET :: ansi.CSI + ansi.RESET + ansi.SGR

View File

@@ -6,8 +6,6 @@ import "core:os"
import "core:path/filepath"
import "core:strings"
import "findr"
SshKeyPair :: struct {
Private: string `json:"private"`,
Public: string `json:"public"`,
@@ -25,16 +23,25 @@ Config :: struct {
config_path: string `json:"-"`,
}
load_config :: proc(config_path: string, allocator := context.allocator) -> (Config, bool) {
// TODO: Should we use context.allocator + defer delete()?
data, read_err := os.read_entire_file_from_path(config_path, context.temp_allocator)
default_config_path :: proc(home: string, allocator := context.allocator) -> string {
path, err := filepath.join([]string{home, ".envr", "config.json"}, allocator)
if err != nil {
panic("Ran out of memory when building config path")
}
return path
}
load_config :: proc(config_path: string) -> (Config, bool) {
data, read_err := os.read_entire_file_from_path(config_path, context.allocator)
if read_err != nil {
fmt.println("No config file found. Please run `envr init` to generate one.")
return Config{}, false
}
defer delete(data)
cfg: Config
err := json.unmarshal(data, &cfg, .JSON5, allocator)
// TODO: use json 5
err := json.unmarshal(data, &cfg)
if err != nil {
fmt.printf("Error parsing config: %v\n", err)
return Config{}, false
@@ -44,104 +51,33 @@ load_config :: proc(config_path: string, allocator := context.allocator) -> (Con
return cfg, true
}
default_config_path :: proc(home: string, allocator := context.allocator) -> string {
path, err := filepath.join([]string{home, ".envr", "config.json"}, allocator)
if err != nil {
panic("Ran out of memory when building config path")
}
return path
}
delete_config :: proc(cfg: ^Config, allocator := context.allocator) {
delete_config :: proc(cfg: ^Config) {
for key in cfg.Keys {
delete(key.Private, allocator)
delete(key.Public, allocator)
delete(key.Private)
delete(key.Public)
}
delete(cfg.Keys)
delete(cfg.ScanConfig.Matcher, allocator)
delete(cfg.ScanConfig.Matcher)
for exclude in cfg.ScanConfig.Exclude {
delete(exclude, allocator)
delete(exclude)
}
delete(cfg.ScanConfig.Exclude)
for include in cfg.ScanConfig.Include {
delete(include, allocator)
delete(include)
}
delete(cfg.ScanConfig.Include)
}
save_config :: proc(cfg: Config, force: bool = false) -> bool {
config_dir := envr_dir(cfg.config_path)
if !os.exists(config_dir) {
mkdir_err := os.make_directory(config_dir)
if mkdir_err != nil {
fmt.printf("Error creating %s directory: %v\n", config_dir, mkdir_err)
return false
}
}
if os.exists(cfg.config_path) && !force {
info, stat_err := os.stat(cfg.config_path, context.temp_allocator)
if stat_err == nil {
defer os.file_info_delete(info, context.temp_allocator)
if info.size > 0 {
fmt.println("Config file already exists. Run again with --force to reinitialize.")
return false
}
}
}
data, marshal_err := json.marshal(
cfg,
{pretty = true, use_spaces = true, spaces = 2},
context.temp_allocator,
)
if marshal_err != nil {
fmt.printf("Error marshaling config: %v\n", marshal_err)
return false
}
write_err := os.write_entire_file(cfg.config_path, data)
if write_err != nil {
fmt.printf("Error writing config: %v\n", write_err)
return false
}
return true
envr_dir :: proc(config_path: string) -> string {
return filepath.dir(config_path)
}
// Caller is responsible for calling delete_config()
new_config :: proc(
private_key_paths: []string,
cfg_path: string = "~/.envr/config.json",
) -> Config {
keys := make([dynamic]SshKeyPair, 0, len(private_key_paths))
for priv in private_key_paths {
// TODO: Is this bad?
priv_key := strings.clone(priv)
pub, _ := strings.concatenate([]string{priv_key, ".pub"})
append(&keys, SshKeyPair{Private = priv_key, Public = pub})
}
exclude := make([dynamic]string, 0, 4)
append(&exclude, strings.clone("*\\.envrc"))
append(&exclude, strings.clone("\\.local/"))
append(&exclude, strings.clone("node_modules"))
append(&exclude, strings.clone("vendor"))
include := make([dynamic]string, 0, 1)
append(&include, strings.clone("~"))
scan_cfg := ScanConfig {
Matcher = strings.clone("\\.env"),
Exclude = exclude,
Include = include,
}
return Config{Keys = keys, ScanConfig = scan_cfg, config_path = cfg_path}
data_path :: proc(config_path: string) -> string {
path, _ := filepath.join([]string{envr_dir(config_path), "data.envr"})
return path
}
find_ssh_private_keys :: proc() -> (keys: [dynamic]string, ok: bool) {
@@ -190,49 +126,115 @@ find_ssh_private_keys :: proc() -> (keys: [dynamic]string, ok: bool) {
return
}
find_git_roots :: proc(
cfg: Config,
allocator := context.temp_allocator,
) -> (
roots: [dynamic]string,
ok: bool,
) {
paths := search_paths(cfg, allocator)
// TODO: Pass allocator to findr
// findr.find_repos(paths[:], &roots, os.get_processor_core_count(), allocator)
findr.find_repos(paths[:], &roots, os.get_processor_core_count())
// Caller is responsible for calling delete_config()
new_config :: proc(
private_key_paths: []string,
cfg_path: string = "~/.envr/config.json",
) -> Config {
keys := make([dynamic]SshKeyPair, 0, len(private_key_paths))
for priv in private_key_paths {
// TODO: Is this bad?
priv_key := strings.clone(priv)
pub, _ := strings.concatenate([]string{priv_key, ".pub"})
append(&keys, SshKeyPair{Private = priv_key, Public = pub})
}
exclude := make([dynamic]string, 0, 4)
append(&exclude, strings.clone("*\\.envrc"))
append(&exclude, strings.clone("\\.local/"))
append(&exclude, strings.clone("node_modules"))
append(&exclude, strings.clone("vendor"))
include := make([dynamic]string, 0, 1)
append(&include, strings.clone("~"))
scan_cfg := ScanConfig {
Matcher = strings.clone("\\.env"),
Exclude = exclude,
Include = include,
}
return Config{Keys = keys, ScanConfig = scan_cfg, config_path = cfg_path}
}
save_config :: proc(cfg: Config, force: bool = false) -> bool {
config_dir := envr_dir(cfg.config_path)
if !os.exists(config_dir) {
mkdir_err := os.make_directory(config_dir)
if mkdir_err != nil {
fmt.printf("Error creating %s directory: %v\n", config_dir, mkdir_err)
return false
}
}
if os.exists(cfg.config_path) && !force {
info, stat_err := os.stat(cfg.config_path, context.allocator)
if stat_err == nil {
defer os.file_info_delete(info, context.allocator)
if info.size > 0 {
fmt.println("Config file already exists. Run again with --force to reinitialize.")
return false
}
}
}
data, marshal_err := json.marshal(cfg, {pretty = true, use_spaces = true, spaces = 2})
if marshal_err != nil {
fmt.printf("Error marshaling config: %v\n", marshal_err)
return false
}
defer delete(data)
write_err := os.write_entire_file(cfg.config_path, data)
if write_err != nil {
fmt.printf("Error writing config: %v\n", write_err)
return false
}
return true
}
search_paths :: proc(cfg: Config) -> (paths: [dynamic]string) {
// TODO: Is this okay?
// TODO: handle error
home, _ := os.user_home_dir(context.temp_allocator)
for include in cfg.ScanConfig.Include {
// TODO: Do we need to manually expand ~/ in odin?
expanded, _ := strings.replace(include, "~", home, 1)
if filepath.is_abs(expanded) {
append(&paths, expanded)
} else {
defer delete(expanded)
resolved, err := filepath.abs(expanded)
if err == nil {
append(&paths, resolved)
}
}
}
return
}
find_git_roots :: proc(cfg: Config) -> (roots: [dynamic]string, ok: bool) {
paths := search_paths(cfg)
for sp in paths {
args := []string{"fd", "-H", "-t", "d", "^\\.git$", sp}
lines, fd_ok := run_fd(args)
if !fd_ok {
return
}
for line in lines {
cleaned, _ := filepath.clean(line)
parent := filepath.dir(cleaned)
cloned, _ := strings.clone(parent)
append(&roots, cloned)
}
}
ok = true
return
}
search_paths :: proc(cfg: Config, allocator := context.allocator) -> [dynamic]string {
// TODO: handle error
home, _ := os.user_home_dir(context.temp_allocator)
paths, _ := new_clone(cfg.ScanConfig.Include, allocator)
for &include in paths {
// TODO: Do we need to manually expand ~/ in odin?
expanded, _ := strings.replace(include, "~", home, 1, allocator)
if filepath.is_abs(expanded) {
include = expanded
} else {
resolved, err := filepath.abs(expanded, allocator)
if err == nil {
include = resolved
}
}
}
return paths^
}
envr_dir :: proc(config_path: string) -> string {
return filepath.dir(config_path)
}
// User is responsible for freeing the path
data_path :: proc(config_path: string, allocator := context.allocator) -> string {
path, _ := filepath.join([]string{envr_dir(config_path), "data.envr"}, allocator)
return path
}

View File

@@ -1,4 +1,3 @@
#+test
package main
import "core:fmt"
@@ -188,10 +187,14 @@ test_search_paths_expands_tilde :: proc(t: ^testing.T) {
cfg := Config {
ScanConfig = ScanConfig{Include = make([dynamic]string, 0, 1)},
}
append(&cfg.ScanConfig.Include, "~")
defer delete(cfg.ScanConfig.Include)
append(&cfg.ScanConfig.Include, "~")
paths := search_paths(cfg, context.temp_allocator)
paths := search_paths(cfg)
defer delete(paths)
for path in paths {
defer delete(path)
}
testing.expect(t, len(paths) == 1, "should have 1 path")
if len(paths) > 0 {

View File

@@ -2,7 +2,6 @@ package main
import "core:fmt"
import "core:mem"
import "core:os"
MAGIC :: "ENVR"
MAGIC_BYTES := [4]u8{u8('E'), u8('N'), u8('V'), u8('R')}
@@ -21,24 +20,85 @@ RecipientEntry :: struct {
EncryptedKey: [CRYPTO_SECRETBOX_KEY_BYTES + CRYPTO_BOX_MAC_BYTES]u8,
}
sodium_initialized: bool
ensure_sodium :: proc() -> bool {
if sodium_initialized {
return true
}
rc := sodium_init()
if rc < 0 {
fmt.println("Error: libsodium initialization failed")
return false
}
sodium_initialized = true
return true
}
X25519Keypair :: struct {
Public: [CRYPTO_BOX_PUBLICKEY_BYTES]u8,
Private: [CRYPTO_BOX_SECRETKEY_BYTES]u8,
}
@(init)
init_sodium :: proc "contextless" () {
if sodium_init() < 0 {
os.exit(1)
ssh_to_x25519 :: proc(keys: []SshKeyPair) -> (pairs: []X25519Keypair, ok: bool) {
if len(keys) == 0 {
return
}
pairs = make([]X25519Keypair, len(keys))
for i in 0 ..< len(keys) {
ssh_kp, parse_ok := parse_ssh_private_key(keys[i].Private)
if !parse_ok {
fmt.printf("Error: failed to parse SSH private key: %s\n", keys[i].Private)
delete(pairs)
return
}
ssh_pub, pub_ok := parse_ssh_public_key(keys[i].Public)
if !pub_ok {
fmt.printf("Error: failed to parse SSH public key: %s\n", keys[i].Public)
delete(pairs)
return
}
pk_rc := crypto_sign_ed25519_pk_to_curve25519(&pairs[i].Public[0], &ssh_pub[0])
if pk_rc != 0 {
fmt.println("Error: failed to convert ed25519 public key to curve25519")
delete(pairs)
return
}
ed25519_sk: [64]u8
for j in 0 ..< 32 {
ed25519_sk[j] = ssh_kp.Private[j]
}
for j in 0 ..< 32 {
ed25519_sk[32 + j] = ssh_kp.Public[j]
}
sk_rc := crypto_sign_ed25519_sk_to_curve25519(&pairs[i].Private[0], &ed25519_sk[0])
if sk_rc != 0 {
fmt.println("Error: failed to convert ed25519 private key to curve25519")
delete(pairs)
return
}
}
ok = true
return
}
// TODO: Optimize performance
encrypt :: proc(plaintext: []u8, keys: []SshKeyPair) -> (ciphertext: []u8, ok: bool) {
x25519_pairs, pairs_ok := ssh_to_x25519(keys, context.temp_allocator)
if !ensure_sodium() {
return
}
x25519_pairs, pairs_ok := ssh_to_x25519(keys)
if !pairs_ok {
return
}
defer delete(x25519_pairs)
sym_key: [CRYPTO_SECRETBOX_KEY_BYTES]u8
randombytes_buf(&sym_key[0], CRYPTO_SECRETBOX_KEY_BYTES)
@@ -47,7 +107,7 @@ encrypt :: proc(plaintext: []u8, keys: []SshKeyPair) -> (ciphertext: []u8, ok: b
randombytes_buf(&main_nonce[0], CRYPTO_SECRETBOX_NONCE_BYTES)
ct_len := len(plaintext) + CRYPTO_SECRETBOX_MAC_BYTES
secret_ct := make([]u8, ct_len, context.temp_allocator)
secret_ct := make([]u8, ct_len)
pt_ptr: [^]u8
if len(plaintext) > 0 {
pt_ptr = &plaintext[0]
@@ -66,7 +126,7 @@ encrypt :: proc(plaintext: []u8, keys: []SshKeyPair) -> (ciphertext: []u8, ok: b
}
num_recipients := u32(len(x25519_pairs))
entries := make([]RecipientEntry, num_recipients, context.temp_allocator)
entries := make([]RecipientEntry, num_recipients)
for i in 0 ..< len(x25519_pairs) {
for j in 0 ..< CRYPTO_BOX_PUBLICKEY_BYTES {
@@ -126,11 +186,17 @@ encrypt :: proc(plaintext: []u8, keys: []SshKeyPair) -> (ciphertext: []u8, ok: b
mem.copy(&ciphertext[pos], &secret_ct[0], ct_len)
delete(entries)
delete(secret_ct)
ok = true
return
}
decrypt :: proc(ciphertext: []u8, keys: []SshKeyPair) -> (plaintext: []u8, ok: bool) {
if !ensure_sodium() {
return
}
if len(ciphertext) < HEADER_SIZE {
fmt.println("Error: ciphertext too short (header)")
return
@@ -174,10 +240,11 @@ decrypt :: proc(ciphertext: []u8, keys: []SshKeyPair) -> (plaintext: []u8, ok: b
enc_nonce: [CRYPTO_BOX_NONCE_BYTES]u8
enc_pub: [CRYPTO_BOX_PUBLICKEY_BYTES]u8
x25519_pairs, pairs_ok := ssh_to_x25519(keys, context.temp_allocator)
x25519_pairs, pairs_ok := ssh_to_x25519(keys)
if !pairs_ok {
return
}
defer delete(x25519_pairs)
found := false
matched_pi := 0
@@ -269,57 +336,3 @@ decrypt :: proc(ciphertext: []u8, keys: []SshKeyPair) -> (plaintext: []u8, ok: b
return
}
ssh_to_x25519 :: proc(
keys: []SshKeyPair,
allocator := context.temp_allocator,
) -> (
[]X25519Keypair,
bool,
) {
if len(keys) == 0 {
return {}, false
}
pairs := make([]X25519Keypair, len(keys), allocator)
for i in 0 ..< len(keys) {
ssh_kp, parse_ok := parse_ssh_private_key(keys[i].Private)
if !parse_ok {
fmt.printf("Error: failed to parse SSH private key: %s\n", keys[i].Private)
delete(pairs)
return pairs, false
}
ssh_pub, pub_ok := parse_ssh_public_key(keys[i].Public)
if !pub_ok {
fmt.printf("Error: failed to parse SSH public key: %s\n", keys[i].Public)
delete(pairs)
return pairs, false
}
pk_rc := crypto_sign_ed25519_pk_to_curve25519(&pairs[i].Public[0], &ssh_pub[0])
if pk_rc != 0 {
fmt.println("Error: failed to convert ed25519 public key to curve25519")
delete(pairs)
return pairs, false
}
ed25519_sk: [64]u8
for j in 0 ..< 32 {
ed25519_sk[j] = ssh_kp.Private[j]
}
for j in 0 ..< 32 {
ed25519_sk[32 + j] = ssh_kp.Public[j]
}
sk_rc := crypto_sign_ed25519_sk_to_curve25519(&pairs[i].Private[0], &ed25519_sk[0])
if sk_rc != 0 {
fmt.println("Error: failed to convert ed25519 private key to curve25519")
delete(pairs)
return pairs, false
}
}
return pairs, true
}

View File

@@ -1,11 +1,9 @@
#+test
package main
import "core:fmt"
import "core:os"
import "core:testing"
CRYPTO_TEST_KEY_DIR :: "fixtures" + os.Path_Separator_String + "keys"
CRYPTO_TEST_KEY_DIR :: "fixtures/keys"
make_test_key_pair :: proc(name: string) -> SshKeyPair {
priv := fmt.tprintf("%s/%s", CRYPTO_TEST_KEY_DIR, name)

760
db.odin
View File

@@ -2,39 +2,35 @@ package main
import "core:crypto/hash"
import "core:encoding/hex"
import "core:encoding/ini"
import "core:encoding/json"
import "core:fmt"
import "core:mem"
import "core:os"
import "core:path/filepath"
import "core:strings"
import "core:time"
import "sqlite"
SyncFlagEnum :: enum {
Noop,
DirUpdated,
Restored,
BackedUp,
Error,
}
SyncFlag :: bit_set[SyncFlagEnum]
SyncError :: enum {
None,
DirMissing,
MultipleDirs,
GitRootFailed,
WriteFailed,
ReadFailed,
DbFailed,
SyncDirection :: enum {
TrustDatabase,
TrustFilesystem,
}
Db :: struct {
conn: sqlite.Db,
// Pointer to the sqlite db
db: ^rawptr,
cfg: Config,
changed: bool,
arena: mem.Dynamic_Arena,
}
EnvFile :: struct {
@@ -45,7 +41,6 @@ EnvFile :: struct {
contents: string,
}
@(deprecated = "call db_close to clean up EnvFiles")
delete_envfile :: proc(f: ^EnvFile) {
delete(f.Path)
for &remote in f.Remotes {
@@ -56,108 +51,61 @@ delete_envfile :: proc(f: ^EnvFile) {
delete(f.contents)
}
db_open :: proc(cfg_path: string) -> (database: Db, ok: bool) {
database = db_init() or_return
database.cfg = load_config(cfg_path, db_allocator(&database)) or_return
// TODO: Use different allocators?
data_path := data_path(database.cfg.config_path, context.temp_allocator)
if os.exists(data_path) {
if ok = db_restore_from_encrypted(&database, data_path); !ok {
sqlite.close(database.conn)
return database, false
}
} else {
// DB was created
database.changed = true
}
return database, true
make_temp_path :: proc() -> string {
ts := time.time_to_unix(time.now())
b: strings.Builder
strings.builder_init(&b)
defer strings.builder_destroy(&b)
fmt.sbprintf(&b, "/tmp/envr-%d-%d.db", os.get_pid(), ts)
return strings.to_string(b)
}
// Creates a database an allocator and fresh, empty table, with zero encryption.
// In production, you most likely want to use `db_open`.
db_init :: proc() -> (database: Db, ok: bool) {
conn: sqlite.Db
rc := sqlite.open(":memory:", &conn)
db_open :: proc(cfg_path: string) -> (Db, bool) {
cfg, ok := load_config(cfg_path)
if !ok {
return Db{}, false
}
data_path := data_path(cfg.config_path)
_, stat_err := os.stat(data_path, context.allocator)
db: ^rawptr
rc := sqlite.db_open(":memory:", &db)
if rc != sqlite.OK {
fmt.printf("Error opening in-memory database: %s\n", sqlite.db_errmsg(conn))
return
fmt.printf("Error opening in-memory database: %s\n", sqlite.db_errmsg(db))
return Db{}, false
}
create_sql: cstring = "CREATE TABLE IF NOT EXISTS envr_env_files (path TEXT PRIMARY KEY NOT NULL, remotes TEXT, sha256 TEXT NOT NULL, contents TEXT NOT NULL)"
rc = sqlite.db_exec(conn, create_sql, nil, nil, nil)
rc = sqlite.db_exec(db, create_sql, nil, nil, nil)
if rc != sqlite.OK {
fmt.printf("Error creating table: %s\n", sqlite.db_errmsg(conn))
sqlite.close(conn)
return
}
database.conn = conn
mem.dynamic_arena_init(&database.arena)
return database, true
}
db_allocator :: proc(db: ^Db) -> mem.Allocator {
return mem.dynamic_arena_allocator(&db.arena)
}
db_restore_from_encrypted :: proc(db: ^Db, data_path: string) -> bool {
encrypted_data, read_err := os.read_entire_file_from_path(data_path, context.temp_allocator)
if read_err != nil {
fmt.printf("Error reading encrypted database: %v\n", read_err)
return false
fmt.printf("Error creating table: %s\n", sqlite.db_errmsg(db))
sqlite.db_close(db)
return Db{}, false
}
// TODO: Use context.temp_allocator
plaintext, dec_ok := decrypt(encrypted_data, db.cfg.Keys[:])
if !dec_ok {
fmt.println("Error: decryption failed")
return false
if stat_err == nil {
if !db_restore_from_encrypted(db, cfg) {
sqlite.db_close(db)
return Db{}, false
}
defer delete(plaintext)
n := i64(len(plaintext))
buf := sqlite.malloc64(n)
if buf == nil {
fmt.println("Error: failed to allocate buffer for deserialization")
return false
}
copy(buf[:len(plaintext)], plaintext)
flags: sqlite.DESERIALIZE_FLAGS = {.FREEONCLOSE, .RESIZEABLE}
rc := sqlite.deserialize(db.conn, "main", buf, n, n, flags)
if rc != sqlite.OK {
sqlite.free(buf)
fmt.printf("Error deserializing database: %s\n", sqlite.db_errmsg(db.conn))
return false
}
return true
return Db{db = db, cfg = cfg, changed = stat_err != nil}, true
}
db_close :: proc(d: ^Db) {
allocator := db_allocator(d)
defer {
sqlite.close(d.conn)
delete_config(&d.cfg, allocator)
mem.dynamic_arena_destroy(&d.arena)
}
defer sqlite.db_close(d.db)
if d.changed {
rc := sqlite.db_exec(d.conn, "VACUUM", nil, nil, nil)
rc := sqlite.db_exec(d.db, "VACUUM", nil, nil, nil)
if rc != sqlite.OK {
fmt.printf("Error vacuuming database: %s\n", sqlite.db_errmsg(d.conn))
fmt.printf("Error vacuuming database: %s\n", sqlite.db_errmsg(d.db))
return
}
sz: i64
data := sqlite.serialize(d.conn, "main", &sz, 0)
data := sqlite.serialize(d.db, "main", &sz, 0)
if data == nil {
fmt.println("Error: failed to serialize database")
return
@@ -165,14 +113,13 @@ db_close :: proc(d: ^Db) {
defer sqlite.free(data)
sqlite_data := data[:sz]
// TODO: PAss allocator chain
encrypted, enc_ok := encrypt(sqlite_data, d.cfg.Keys[:])
if !enc_ok {
fmt.println("Error: encryption failed")
return
}
data_path := data_path(d.cfg.config_path, allocator)
data_path := data_path(d.cfg.config_path)
envr_d := envr_dir(d.cfg.config_path)
os.mkdir_all(envr_d)
@@ -187,33 +134,36 @@ db_close :: proc(d: ^Db) {
}
}
// Results will be freed when `db_close` is called.
db_list :: proc(d: ^Db) -> ([]EnvFile, bool) {
stmt: sqlite.Stmt
// Caller is responsible for calling:
// ```odin
// delete(results)
// for &result in results {
// delete(&result)
// }
// ```
db_list :: proc(d: ^Db, allocator := context.allocator) -> (results: [dynamic]EnvFile, ok: bool) {
stmt: ^rawptr
rc := sqlite.prepare_v2(
d.conn,
d.db,
"SELECT path, remotes, sha256, contents FROM envr_env_files",
-1,
&stmt,
nil,
)
if rc != sqlite.OK {
fmt.printf("Error preparing query: %s\n", sqlite.db_errmsg(d.conn))
return []EnvFile{}, false
fmt.printf("Error preparing query: %s\n", sqlite.db_errmsg(d.db))
return
}
defer sqlite.finalize(stmt)
allocator := db_allocator(d)
results := make([dynamic]EnvFile, 0, 10, allocator)
for {
rc = sqlite.step(stmt)
if rc == sqlite.DONE {
break
}
if rc != sqlite.ROW {
fmt.printf("Error stepping query: %s\n", sqlite.db_errmsg(d.conn))
#no_bounds_check return results[:], false
fmt.printf("Error stepping query: %s\n", sqlite.db_errmsg(d.db))
return
}
remotes_json := string(sqlite.column_text(stmt, 1))
@@ -235,149 +185,115 @@ db_list :: proc(d: ^Db) -> ([]EnvFile, bool) {
)
}
#no_bounds_check return results[:], true
ok = true
return
}
// TODO: Should we use context.temp_allocator for proc scoped lifetimes?
db_insert :: proc(d: ^Db, file: EnvFile) -> bool {
remotes_json, marshal_err := json.marshal(file.Remotes, allocator = context.temp_allocator)
if marshal_err != nil {
fmt.printf("Error marshaling remotes: %v\n", marshal_err)
db_restore_from_encrypted :: proc(db: ^rawptr, cfg: Config) -> bool {
encrypted_data, read_err := os.read_entire_file_from_path(
data_path(cfg.config_path),
context.allocator,
)
defer delete(encrypted_data)
if read_err != nil {
fmt.printf("Error reading encrypted database: %v\n", read_err)
return false
}
sql: cstring =
"INSERT OR REPLACE INTO " +
"envr_env_files (path, remotes, sha256, contents) VALUES (?, ?, ?, ?)"
stmt: sqlite.Stmt
rc := sqlite.prepare_v2(d.conn, sql, -1, &stmt, nil)
plaintext, dec_ok := decrypt(encrypted_data, cfg.Keys[:])
if !dec_ok {
fmt.println("Error: decryption failed")
return false
}
defer delete(plaintext)
n := i64(len(plaintext))
buf := sqlite.malloc64(n)
if buf == nil {
fmt.println("Error: failed to allocate buffer for deserialization")
return false
}
copy(buf[:len(plaintext)], plaintext)
rc := sqlite.deserialize(
db,
"main",
buf,
n,
n,
sqlite.DESERIALIZE_FREEONCLOSE | sqlite.DESERIALIZE_RESIZEABLE,
)
if rc != sqlite.OK {
fmt.printf("Error preparing insert: %s\n", sqlite.db_errmsg(d.conn))
return false
}
defer sqlite.finalize(stmt)
// TODO: deal with elsewhere?
cpath := to_cstring(file.Path)
defer delete(cpath)
rc = sqlite.bind_text(stmt, 1, cpath, -1, nil)
if rc != sqlite.OK {
fmt.printf("Error binding path: %s\n", sqlite.db_errmsg(d.conn))
sqlite.free(buf)
fmt.printf("Error deserializing database: %s\n", sqlite.db_errmsg(db))
return false
}
cremotes := to_cstring(string(remotes_json))
defer delete(cremotes)
rc = sqlite.bind_text(stmt, 2, cremotes, -1, nil)
if rc != sqlite.OK {
fmt.printf("Error binding remotes: %s\n", sqlite.db_errmsg(d.conn))
return false
}
csha := to_cstring(file.Sha256)
defer delete(csha)
rc = sqlite.bind_text(stmt, 3, csha, -1, nil)
if rc != sqlite.OK {
fmt.printf("Error binding sha256: %s\n", sqlite.db_errmsg(d.conn))
return false
}
ccontents := to_cstring(file.contents)
defer delete(ccontents)
rc = sqlite.bind_text(stmt, 4, ccontents, -1, nil)
if rc != sqlite.OK {
fmt.printf("Error binding contents: %s\n", sqlite.db_errmsg(d.conn))
return false
}
rc = sqlite.step(stmt)
if rc != sqlite.DONE {
fmt.printf("Error inserting: %s\n", sqlite.db_errmsg(d.conn))
return false
}
d.changed = true
return true
}
// Result will be freed when `db_close` is called.
db_fetch :: proc(d: ^Db, path: string) -> (EnvFile, bool) {
sql: cstring = "SELECT path, remotes, sha256, contents FROM envr_env_files WHERE path = ?"
stmt: sqlite.Stmt
rc := sqlite.prepare_v2(d.conn, sql, -1, &stmt, nil)
if rc != sqlite.OK {
fmt.printf("Error preparing fetch: %s\n", sqlite.db_errmsg(d.conn))
return EnvFile{}, false
}
defer sqlite.finalize(stmt)
allocator := db_allocator(d)
get_git_remotes :: proc(dir: string) -> [dynamic]string {
remotes: [dynamic]string
remote_set: map[string]bool
cpath := to_cstring(path, allocator)
defer delete(cpath, allocator)
rc = sqlite.bind_text(stmt, 1, cpath, -1, nil)
if rc != sqlite.OK {
fmt.printf("Error binding path: %s\n", sqlite.db_errmsg(d.conn))
return EnvFile{}, false
}
rc = sqlite.step(stmt)
if rc == sqlite.DONE {
fmt.printf("No file found with path: %s\n", path)
return EnvFile{}, false
}
if rc != sqlite.ROW {
fmt.printf("Error fetching: %s\n", sqlite.db_errmsg(d.conn))
return EnvFile{}, false
b: strings.Builder
strings.builder_init(&b)
defer strings.builder_destroy(&b)
fmt.sbprintf(&b, "%s-git-remotes", make_temp_path())
tmp_path := strings.to_string(b)
tmp_file, tmp_err := os.open(tmp_path, os.O_CREATE | os.O_WRONLY | os.O_TRUNC)
if tmp_err != nil {
return remotes
}
remotes_json := string(sqlite.column_text(stmt, 1))
remotes: [dynamic]string = ---
if len(remotes_json) > 0 {
json.unmarshal_string(remotes_json, &remotes, allocator = allocator)
args := []string{"git", "remote", "-v"}
desc := os.Process_Desc {
command = args,
stdout = tmp_file,
stderr = nil,
working_dir = dir,
}
file_path := clone_cstring(sqlite.column_text(stmt, 0), allocator)
return EnvFile {
Path = file_path,
Dir = filepath.dir(file_path),
Remotes = remotes,
Sha256 = clone_cstring(sqlite.column_text(stmt, 2), allocator),
contents = clone_cstring(sqlite.column_text(stmt, 3), allocator),
},
true
}
db_delete :: proc(d: ^Db, path: string) -> bool {
sql: cstring = "DELETE FROM envr_env_files WHERE path = ?"
stmt: sqlite.Stmt
rc := sqlite.prepare_v2(d.conn, sql, -1, &stmt, nil)
if rc != sqlite.OK {
fmt.printf("Error preparing delete: %s\n", sqlite.db_errmsg(d.conn))
return false
}
defer sqlite.finalize(stmt)
cpath := to_cstring(path)
defer delete(cpath)
rc = sqlite.bind_text(stmt, 1, cpath, -1, nil)
if rc != sqlite.OK {
fmt.printf("Error binding path: %s\n", sqlite.db_errmsg(d.conn))
return false
}
rc = sqlite.step(stmt)
if rc != sqlite.DONE {
fmt.printf("Error deleting: %s\n", sqlite.db_errmsg(d.conn))
return false
p, start_err := os.process_start(desc)
os.close(tmp_file)
if start_err != nil {
os.remove(tmp_path)
return remotes
}
if sqlite.changes(d.conn) == 0 {
fmt.printf("No file found with path: %s\n", path)
return false
state, wait_err := os.process_wait(p)
if wait_err != nil || state.exit_code != 0 {
os.remove(tmp_path)
return remotes
}
d.changed = true
return true
data, read_err := os.read_entire_file_from_path(tmp_path, context.allocator)
defer delete(data)
os.remove(tmp_path)
if read_err != nil {
return remotes
}
lines := strings.split(string(data), "\n")
for &line in lines {
line = strings.trim_space(line)
if len(line) == 0 {
continue
}
parts := strings.fields(line)
if len(parts) >= 2 {
remote_set[parts[1]] = true
}
}
for remote, _ in remote_set {
cloned, _ := strings.clone(remote)
append(&remotes, cloned)
}
return remotes
}
new_env_file :: proc(path: string) -> (EnvFile, bool) {
@@ -389,8 +305,7 @@ new_env_file :: proc(path: string) -> (EnvFile, bool) {
dir := filepath.dir(abs_path)
// TODO: Should we use the db allocator here?
remotes := get_git_remotes(dir, context.allocator)
remotes := get_git_remotes(dir)
data, read_err := os.read_entire_file_from_path(abs_path, context.allocator)
defer delete(data)
@@ -413,146 +328,143 @@ new_env_file :: proc(path: string) -> (EnvFile, bool) {
true
}
// Reconciles `f` with the filesystem and persists changes to the database.
db_sync :: proc(d: ^Db, f: ^EnvFile) -> (SyncFlag, SyncError) {
allocator := db_allocator(d)
result: SyncFlag = {}
old_path := f.Path
if !os.exists(f.Dir) {
moved, err := try_move_dir(d, f, allocator)
if !moved {
return {}, err
}
result += {.DirUpdated}
}
if !os.exists(f.Path) {
write_err := os.write_entire_file(f.Path, f.contents)
if write_err != nil {
fmt.eprintf("db_sync: failed to write %s: %v\n", f.Path, write_err)
return result, .WriteFailed
}
if !db_persist(d, f, old_path) {
return result, .DbFailed
}
return result + {.Restored}, .None
}
data, read_err := os.read_entire_file_from_path(f.Path, allocator)
if read_err != nil {
fmt.eprintf("db_sync: failed to read %s: %v\n", f.Path, read_err)
return result, .ReadFailed
}
digest := hash.hash_bytes(hash.Algorithm.SHA256, data, context.temp_allocator)
hex_bytes, hex_err := hex.encode(digest, allocator)
if hex_err != nil {
fmt.eprintf("db_sync: failed to encode hash for %s: %v\n", f.Path, hex_err)
return result, .ReadFailed
}
current_sha := string(hex_bytes)
if current_sha == f.Sha256 {
if !db_persist(d, f, old_path) {
return result, .DbFailed
}
return result, .None
}
f.contents = string(data)
f.Sha256 = current_sha
if !db_persist(d, f, old_path) {
return result, .DbFailed
}
return result + {.BackedUp}, .None
}
db_persist :: proc(d: ^Db, f: ^EnvFile, old_path: string) -> bool {
if f.Path != old_path {
if !db_delete(d, old_path) {
db_insert :: proc(d: ^Db, file: EnvFile) -> bool {
remotes_json, marshal_err := json.marshal(file.Remotes)
if marshal_err != nil {
fmt.printf("Error marshaling remotes: %v\n", marshal_err)
return false
}
}
return db_insert(d, f^)
}
defer delete(remotes_json)
try_move_dir :: proc(d: ^Db, f: ^EnvFile, allocator: mem.Allocator) -> (bool, SyncError) {
roots, ok := find_git_roots(d.cfg)
if !ok {
return false, .GitRootFailed
sql: cstring =
"INSERT OR REPLACE INTO " +
"envr_env_files (path, remotes, sha256, contents) VALUES (?, ?, ?, ?)"
stmt: ^rawptr
rc := sqlite.prepare_v2(d.db, sql, -1, &stmt, nil)
if rc != sqlite.OK {
fmt.printf("Error preparing insert: %s\n", sqlite.db_errmsg(d.db))
return false
}
defer {
for root in roots {
delete(root)
}
delete(roots)
defer sqlite.finalize(stmt)
// TODO: deal with elsewhere?
cpath := to_cstring(file.Path)
defer delete(cpath)
rc = sqlite.bind_text(stmt, 1, cpath, -1, nil)
if rc != sqlite.OK {
fmt.printf("Error binding path: %s\n", sqlite.db_errmsg(d.db))
return false
}
match_count := 0
matched_dir: string
for root in roots {
remotes := get_git_remotes(root, context.temp_allocator)
if shares_remote(f, remotes[:]) {
match_count += 1
matched_dir = root
}
cremotes := to_cstring(string(remotes_json))
defer delete(cremotes)
rc = sqlite.bind_text(stmt, 2, cremotes, -1, nil)
if rc != sqlite.OK {
fmt.printf("Error binding remotes: %s\n", sqlite.db_errmsg(d.db))
return false
}
switch match_count {
case 0:
return false, .DirMissing
case 1:
f.Dir, _ = strings.clone(matched_dir, allocator)
base := filepath.base(f.Path)
new_path, _ := filepath.join({f.Dir, base}, allocator)
f.Path = new_path
f.Remotes = get_git_remotes(f.Dir, allocator)
return true, .None
case:
return false, .MultipleDirs
csha := to_cstring(file.Sha256)
defer delete(csha)
rc = sqlite.bind_text(stmt, 3, csha, -1, nil)
if rc != sqlite.OK {
fmt.printf("Error binding sha256: %s\n", sqlite.db_errmsg(d.db))
return false
}
}
shares_remote :: proc(f: ^EnvFile, remotes: []string) -> bool {
for r1 in f.Remotes {
for r2 in remotes {
if r1 == r2 {
ccontents := to_cstring(file.contents)
defer delete(ccontents)
rc = sqlite.bind_text(stmt, 4, ccontents, -1, nil)
if rc != sqlite.OK {
fmt.printf("Error binding contents: %s\n", sqlite.db_errmsg(d.db))
return false
}
rc = sqlite.step(stmt)
if rc != sqlite.DONE {
fmt.printf("Error inserting: %s\n", sqlite.db_errmsg(d.db))
return false
}
d.changed = true
return true
}
}
}
return false
}
get_git_remotes :: proc(dir: string, allocator: mem.Allocator) -> [dynamic]string {
config_path, _ := filepath.join({dir, ".git", "config"}, context.temp_allocator)
// TODO: Handle error
m, _, read_ok := ini.load_map_from_path(config_path, context.temp_allocator)
if !read_ok {
return nil
db_fetch :: proc(d: ^Db, path: string, allocator := context.allocator) -> (EnvFile, bool) {
sql: cstring = "SELECT path, remotes, sha256, contents FROM envr_env_files WHERE path = ?"
stmt: ^rawptr
rc := sqlite.prepare_v2(d.db, sql, -1, &stmt, nil)
if rc != sqlite.OK {
fmt.printf("Error preparing fetch: %s\n", sqlite.db_errmsg(d.db))
return EnvFile{}, false
}
defer sqlite.finalize(stmt)
cpath := to_cstring(path, allocator)
defer delete(cpath, allocator)
rc = sqlite.bind_text(stmt, 1, cpath, -1, nil)
if rc != sqlite.OK {
fmt.printf("Error binding path: %s\n", sqlite.db_errmsg(d.db))
return EnvFile{}, false
}
rc = sqlite.step(stmt)
if rc == sqlite.DONE {
fmt.printf("No file found with path: %s\n", path)
return EnvFile{}, false
}
if rc != sqlite.ROW {
fmt.printf("Error fetching: %s\n", sqlite.db_errmsg(d.db))
return EnvFile{}, false
}
remotes := make([dynamic]string, 0, 1, allocator)
for section_name, section in m {
if strings.has_prefix(section_name, "remote ") {
if url, ok := section["url"]; ok {
found := false
for r in remotes {
if r == url {found = true; break}
}
if !found {
// FIXME: Currently leaks when adding a file with envr scan
cloned, _ := strings.clone(url, allocator)
append(&remotes, cloned)
}
}
}
remotes_json := string(sqlite.column_text(stmt, 1))
remotes: [dynamic]string = ---
if len(remotes_json) > 0 {
json.unmarshal_string(remotes_json, &remotes, allocator = allocator)
}
return remotes
file_path := clone_cstring(sqlite.column_text(stmt, 0))
return EnvFile {
Path = file_path,
Dir = filepath.dir(file_path),
Remotes = remotes,
Sha256 = clone_cstring(sqlite.column_text(stmt, 2), allocator),
contents = clone_cstring(sqlite.column_text(stmt, 3), allocator),
},
true
}
db_delete :: proc(d: ^Db, path: string) -> bool {
sql: cstring = "DELETE FROM envr_env_files WHERE path = ?"
stmt: ^rawptr
rc := sqlite.prepare_v2(d.db, sql, -1, &stmt, nil)
if rc != sqlite.OK {
fmt.printf("Error preparing delete: %s\n", sqlite.db_errmsg(d.db))
return false
}
defer sqlite.finalize(stmt)
cpath := to_cstring(path)
defer delete(cpath)
rc = sqlite.bind_text(stmt, 1, cpath, -1, nil)
if rc != sqlite.OK {
fmt.printf("Error binding path: %s\n", sqlite.db_errmsg(d.db))
return false
}
rc = sqlite.step(stmt)
if rc != sqlite.DONE {
fmt.printf("Error deleting: %s\n", sqlite.db_errmsg(d.db))
return false
}
if sqlite.changes(d.db) == 0 {
fmt.printf("No file found with path: %s\n", path)
return false
}
d.changed = true
return true
}
to_cstring :: proc {
@@ -569,7 +481,6 @@ string_to_cstring :: proc(s: string, allocator := context.allocator) -> cstring
return cs
}
// Unless an explicit allocator is passed, caller is responsible for freeing the result
clone_cstring :: proc(c: cstring, allocator := context.allocator) -> string {
str, err := strings.clone_from_cstring(c, allocator)
if err != nil {
@@ -581,3 +492,146 @@ clone_cstring :: proc(c: cstring, allocator := context.allocator) -> string {
return str
}
db_update_required :: proc(status: SyncFlag) -> bool {
return .BackedUp in status || .DirUpdated in status
}
shares_remote :: proc(f: ^EnvFile, remotes: []string) -> bool {
for r1 in f.Remotes {
for r2 in remotes {
if r1 == r2 {
return true
}
}
}
return false
}
update_dir :: proc(f: ^EnvFile, new_dir: string) {
f.Dir = new_dir
base := filepath.base(f.Path)
new_path, _ := strings.concatenate({new_dir, "/", base})
f.Path = new_path
f.Remotes = get_git_remotes(new_dir)
}
find_moved_dirs :: proc(d: ^Db, f: ^EnvFile) -> ([dynamic]string, bool) {
feats := check_features()
if .Fd not_in feats || .Git not_in feats {
fmt.println("Error: fd and git are required for moved dir detection")
return {}, false
}
roots, roots_ok := find_git_roots(d.cfg)
if !roots_ok {
return {}, false
}
moved: [dynamic]string
for root in roots {
remotes := get_git_remotes(root)
if shares_remote(f, remotes[:]) {
cloned, _ := strings.clone(root)
append(&moved, cloned)
}
}
return moved, true
}
db_sync :: proc(d: ^Db, f: ^EnvFile) -> (SyncFlag, string) {
return env_file_sync(f, .TrustFilesystem, d)
}
// If SyncFlag is .BackedUp, Caller is responsible for calling delete on f.contents and f.Sha256
env_file_sync :: proc(f: ^EnvFile, dir: SyncDirection, d: ^Db) -> (SyncFlag, string) {
result: SyncFlag = {}
_, stat_err := os.stat(f.Dir, context.allocator)
if stat_err != nil {
moved_dirs: [dynamic]string
if d != nil {
dirs, dirs_ok := find_moved_dirs(d, f)
if !dirs_ok {
return {.Error}, "failed to find moved dirs"
}
moved_dirs = dirs
}
if len(moved_dirs) == 0 {
return {.Error}, "directory missing"
} else if len(moved_dirs) == 1 {
update_dir(f, moved_dirs[0])
result = {.DirUpdated}
} else {
return {.Error}, "multiple directories found"
}
}
_, file_stat_err := os.stat(f.Path, context.allocator)
if file_stat_err != nil {
write_err := os.write_entire_file(f.Path, f.contents)
if write_err != nil {
msg, _ := strings.concatenate({"failed to write file: ", fmt.tprintf("%v", write_err)})
return {.Error}, msg
}
return result + {.Restored}, ""
}
data, read_err := os.read_entire_file_from_path(f.Path, context.allocator)
if read_err != nil {
msg, _ := strings.concatenate(
{"failed to read file for SHA comparison: ", fmt.tprintf("%v", read_err)},
)
return {.Error}, msg
}
digest := hash.hash_bytes(hash.Algorithm.SHA256, data)
// TODO: Handle error
hex_bytes, _ := hex.encode(digest)
current_sha := string(hex_bytes)
if current_sha == f.Sha256 {
return result, ""
}
switch dir {
case .TrustDatabase:
write_err := os.write_entire_file(f.Path, f.contents)
if write_err != nil {
msg, _ := strings.concatenate({"failed to write file: ", fmt.tprintf("%v", write_err)})
return {.Error}, msg
}
return result + {.Restored}, ""
case .TrustFilesystem:
if !env_file_backup(f) {
return {.Error}, "failed to backup file"
}
return result + {.BackedUp}, ""
}
return result, ""
}
// Loads the contents of the the file at f.Path into f.contents
//
// Caller is responsible for calling delete on f.contents and f.Sha256
env_file_backup :: proc(f: ^EnvFile) -> bool {
data, read_err := os.read_entire_file_from_path(f.Path, context.allocator)
if read_err != nil {
fmt.printf("Error reading file %s: %v\n", f.Path, read_err)
return false
}
f.contents = string(data)
digest := hash.hash_bytes(hash.Algorithm.SHA256, data, context.temp_allocator)
hex_bytes, alloc_err := hex.encode(digest)
if alloc_err != nil {
fmt.printf("Error generating hash for file %s: %v\n", f.Path, alloc_err)
return false
}
f.Sha256 = string(hex_bytes)
return true
}

View File

@@ -1,4 +1,3 @@
#+test
package main
import "core:fmt"
@@ -165,13 +164,13 @@ test_decrypt_then_deserialize_sqlite :: proc(t: ^testing.T) {
}
defer delete(plaintext)
mem_db: sqlite.Db
rc := sqlite.open(":memory:", &mem_db)
mem_db: ^rawptr
rc := sqlite.db_open(":memory:", &mem_db)
testing.expectf(t, rc == sqlite.OK, "failed to open in-memory db")
if rc != sqlite.OK {
return
}
defer sqlite.close(mem_db)
defer sqlite.db_close(mem_db)
n := i64(len(plaintext))
buf := sqlite.malloc64(n)
@@ -179,7 +178,14 @@ test_decrypt_then_deserialize_sqlite :: proc(t: ^testing.T) {
if buf == nil do return
copy(buf[:len(plaintext)], plaintext)
rc = sqlite.deserialize(mem_db, "main", buf, n, n, {.FREEONCLOSE, .RESIZEABLE})
rc = sqlite.deserialize(
mem_db,
"main",
buf,
n,
n,
sqlite.DESERIALIZE_FREEONCLOSE | sqlite.DESERIALIZE_RESIZEABLE,
)
testing.expect(t, rc == sqlite.OK, "deserialize should succeed")
if rc != sqlite.OK {
sqlite.free(buf)
@@ -187,7 +193,7 @@ test_decrypt_then_deserialize_sqlite :: proc(t: ^testing.T) {
}
sql: cstring = "SELECT path FROM envr_env_files"
stmt: sqlite.Stmt
stmt: ^rawptr
rc = sqlite.prepare_v2(mem_db, sql, -1, &stmt, nil)
testing.expect(t, rc == sqlite.OK, "prepare failed")
if rc != sqlite.OK {
@@ -303,6 +309,7 @@ test_ssh_key_parse_from_fixtures :: proc(t: ^testing.T) {
if !x_ok {
return
}
defer delete(x25519_pairs)
testing.expect(t, len(x25519_pairs) == 1, "should have 1 x25519 keypair")
}

View File

@@ -1,8 +1,5 @@
#+test
package main
import "core:crypto/hash"
import "core:encoding/hex"
import "core:fmt"
import "core:os"
import "core:path/filepath"
@@ -11,13 +8,30 @@ import "core:testing"
import "sqlite"
make_test_db :: proc() -> (Db, bool) {
db: ^rawptr
rc := sqlite.db_open(":memory:", &db)
if rc != sqlite.OK {
return Db{}, false
}
create_sql: cstring = "CREATE TABLE IF NOT EXISTS envr_env_files (path TEXT PRIMARY KEY NOT NULL, remotes TEXT, sha256 TEXT NOT NULL, contents TEXT NOT NULL)"
rc = sqlite.db_exec(db, create_sql, nil, nil, nil)
if rc != sqlite.OK {
sqlite.db_close(db)
return Db{}, false
}
return Db{db = db}, true
}
make_test_env_file :: proc(path, sha, contents: string, remotes: []string = {}) -> EnvFile {
f := EnvFile {
Path = path,
Dir = "",
Sha256 = sha,
contents = contents,
Remotes = make([dynamic]string, 0, len(remotes), context.temp_allocator),
Remotes = make([dynamic]string, 0, len(remotes)),
}
for r in remotes {
append(&f.Remotes, r)
@@ -27,10 +41,10 @@ make_test_env_file :: proc(path, sha, contents: string, remotes: []string = {})
@(test)
test_db_insert_and_fetch :: proc(t: ^testing.T) {
d, ok := db_init()
d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer db_close(&d)
defer sqlite.db_close(d.db)
path := "/project/.env"
sha := "abc123"
@@ -42,7 +56,7 @@ test_db_insert_and_fetch :: proc(t: ^testing.T) {
testing.expect(t, db_insert(&d, f), "insert should succeed")
fetched, fetch_ok := db_fetch(&d, "/project/.env")
// defer delete_envfile(&fetched)
defer delete_envfile(&fetched)
testing.expect(t, fetch_ok, "fetch should succeed")
if !fetch_ok do return
@@ -55,10 +69,10 @@ test_db_insert_and_fetch :: proc(t: ^testing.T) {
@(test)
test_db_fetch_missing :: proc(t: ^testing.T) {
d, ok := db_init()
d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer db_close(&d)
defer sqlite.db_close(d.db)
_, fetch_ok := db_fetch(&d, "/nonexistent/.env")
testing.expect(t, !fetch_ok, "fetch missing should return false")
@@ -66,9 +80,10 @@ test_db_fetch_missing :: proc(t: ^testing.T) {
@(test)
test_db_insert_or_replace :: proc(t: ^testing.T) {
d, ok := db_init()
defer db_close(&d)
d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer sqlite.db_close(d.db)
f1 := make_test_env_file("/project/.env", "sha1", "KEY=old")
defer delete(f1.Remotes)
@@ -80,13 +95,18 @@ test_db_insert_or_replace :: proc(t: ^testing.T) {
results, list_ok := db_list(&d)
testing.expect(t, list_ok, "list should succeed")
if !list_ok do return
defer delete(results)
for &result in results {
defer delete_envfile(&result)
}
testing.expect(t, len(results) == 1, "should have 1 row, not 2")
fetched, fetch_ok := db_fetch(&d, "/project/.env")
testing.expect(t, fetch_ok, "fetch should succeed")
if !fetch_ok do return
// defer delete_envfile(&fetched)
defer delete_envfile(&fetched)
testing.expect_value(t, fetched.contents, "KEY=new")
testing.expect_value(t, fetched.Sha256, "sha2")
@@ -94,10 +114,10 @@ test_db_insert_or_replace :: proc(t: ^testing.T) {
@(test)
test_db_delete_existing :: proc(t: ^testing.T) {
d, ok := db_init()
d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer db_close(&d)
defer sqlite.db_close(d.db)
f := make_test_env_file("/project/.env", "sha", "KEY=val")
defer delete(f.Remotes)
@@ -111,19 +131,20 @@ test_db_delete_existing :: proc(t: ^testing.T) {
@(test)
test_db_delete_missing :: proc(t: ^testing.T) {
d, ok := db_init()
d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer db_close(&d)
defer sqlite.db_close(d.db)
testing.expect(t, !db_delete(&d, "/nonexistent/.env"), "delete missing should return false")
}
@(test)
test_db_list_multiple :: proc(t: ^testing.T) {
d, ok := db_init()
d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db")
defer db_close(&d)
if !ok do return
defer sqlite.db_close(d.db)
f1 := make_test_env_file("/proj1/.env", "sha1", "A=1", []string{"git@github.com:a/repo.git"})
defer delete(f1.Remotes)
@@ -137,27 +158,36 @@ test_db_list_multiple :: proc(t: ^testing.T) {
results, list_ok := db_list(&d)
testing.expect(t, list_ok, "list should succeed")
if !list_ok do return
defer delete(results)
defer {
for &result in results {
delete_envfile(&result)
}
}
testing.expect_value(t, len(results), 3)
}
@(test)
test_db_list_empty :: proc(t: ^testing.T) {
d, ok := db_init()
d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db")
defer db_close(&d)
if !ok do return
defer sqlite.db_close(d.db)
results, list_ok := db_list(&d)
testing.expect(t, list_ok, "list should succeed on empty db")
testing.expect(t, len(results) == 0, "should have 0 rows")
if list_ok do delete(results)
}
@(test)
test_db_insert_sets_changed :: proc(t: ^testing.T) {
d, ok := db_init()
d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer db_close(&d)
defer sqlite.db_close(d.db)
testing.expect(t, !d.changed, "changed should start false")
@@ -170,10 +200,10 @@ test_db_insert_sets_changed :: proc(t: ^testing.T) {
@(test)
test_db_delete_sets_changed :: proc(t: ^testing.T) {
d, ok := db_init()
d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer db_close(&d)
defer sqlite.db_close(d.db)
f := make_test_env_file("/project/.env", "sha", "KEY=val")
defer delete(f.Remotes)
@@ -186,17 +216,17 @@ test_db_delete_sets_changed :: proc(t: ^testing.T) {
@(test)
test_db_serialize :: proc(t: ^testing.T) {
d, ok := db_init()
d, ok := make_test_db()
testing.expect(t, ok, "failed to create test db")
if !ok do return
defer db_close(&d)
defer sqlite.db_close(d.db)
f := make_test_env_file("/project/.env", "sha", "KEY=val")
defer delete(f.Remotes)
db_insert(&d, f)
sz: i64
data := sqlite.serialize(d.conn, "main", &sz, 0)
data := sqlite.serialize(d.db, "main", &sz, 0)
testing.expect(t, data != nil, "serialize should return non-nil")
if data == nil do return
defer sqlite.free(data)
@@ -204,6 +234,37 @@ test_db_serialize :: proc(t: ^testing.T) {
testing.expect(t, sz > 0, "serialized size should be > 0")
}
@(test)
test_db_update_required_noop :: proc(t: ^testing.T) {
testing.expect(t, !db_update_required({}), "Noop should not require update")
}
@(test)
test_db_update_required_backed_up :: proc(t: ^testing.T) {
testing.expect(t, db_update_required({.BackedUp}), "BackedUp should require update")
}
@(test)
test_db_update_required_dir_updated :: proc(t: ^testing.T) {
testing.expect(t, db_update_required({.DirUpdated}), "DirUpdated should require update")
}
@(test)
test_db_update_required_restored :: proc(t: ^testing.T) {
testing.expect(t, !db_update_required({.Restored}), "Restored alone should not require update")
}
@(test)
test_db_update_required_error :: proc(t: ^testing.T) {
testing.expect(t, !db_update_required({.Error}), "Error alone should not require update")
}
@(test)
test_db_update_required_combined :: proc(t: ^testing.T) {
combined := SyncFlag{.DirUpdated, .Restored}
testing.expect(t, db_update_required(combined), "DirUpdated|Restored should require update")
}
@(test)
test_shares_remote_overlap :: proc(t: ^testing.T) {
f := EnvFile {
@@ -258,81 +319,11 @@ test_shares_remote_both_empty :: proc(t: ^testing.T) {
testing.expect(t, !shares_remote(&f, remotes), "both empty should not share")
}
delete_remotes :: proc(remotes: [dynamic]string) {
for &r in remotes {
delete(r)
}
delete(remotes)
}
@(test)
test_get_git_remotes_single :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-remotes-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
git_dir := fmt.tprintf("%s/.git", base)
os.mkdir_all(git_dir)
config_content := "[core]\n\trepositoryformatversion = 0\n[remote \"origin\"]\n\turl = git@github.com:user/repo.git\n\tfetch = +refs/heads/*:refs/remotes/origin/*\n"
config_path := fmt.tprintf("%s/config", git_dir)
err := os.write_entire_file(config_path, transmute([]u8)config_content)
testing.expect(t, err == nil, "should write .git/config")
remotes := get_git_remotes(base, context.temp_allocator)
testing.expect(t, len(remotes) == 1, "should find 1 remote")
if len(remotes) != 1 do return
testing.expect_value(t, remotes[0], "git@github.com:user/repo.git")
}
@(test)
test_get_git_remotes_multiple :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-remotes-multi-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
git_dir := fmt.tprintf("%s/.git", base)
os.mkdir_all(git_dir)
config_content := "[remote \"origin\"]\n\turl = git@github.com:user/repo.git\n[remote \"upstream\"]\n\turl = https://gitlab.com/upstream/repo.git\n"
config_path := fmt.tprintf("%s/config", git_dir)
err := os.write_entire_file(config_path, transmute([]u8)config_content)
testing.expect(t, err == nil, "should write .git/config")
remotes := get_git_remotes(base, context.temp_allocator)
testing.expect(t, len(remotes) == 2, "should find 2 remotes")
}
@(test)
test_get_git_remotes_no_config :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-remotes-none-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
remotes := get_git_remotes(base, context.temp_allocator)
testing.expect(t, len(remotes) == 0, "should return empty when no .git/config")
}
@(test)
test_get_git_remotes_no_remotes :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-remotes-empty-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
git_dir := fmt.tprintf("%s/.git", base)
os.mkdir_all(git_dir)
config_content := "[core]\n\trepositoryformatversion = 0\n\tbare = false\n"
config_path := fmt.tprintf("%s/config", git_dir)
err := os.write_entire_file(config_path, transmute([]u8)config_content)
testing.expect(t, err == nil, "should write .git/config")
remotes := get_git_remotes(base, context.temp_allocator)
testing.expect(t, len(remotes) == 0, "should return empty when no remote sections")
test_make_temp_path_format :: proc(t: ^testing.T) {
p := make_temp_path()
testing.expect(t, strings.has_suffix(p, ".db"), "should end with .db")
testing.expect(t, strings.contains(p, fmt.tprintf("%d", os.get_pid())), "should contain PID")
}
@(test)
@@ -365,206 +356,45 @@ test_new_env_file_missing :: proc(t: ^testing.T) {
}
@(test)
test_closing_db_has_no_leaks :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-leak-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
cfg_path, err := filepath.join([]string{base, "config.json"}, context.temp_allocator)
testing.expect(t, err == nil, "cfgPath should build successfully")
{
cfg := new_config([]string{"fixtures/keys/insecure-test-key"}, cfg_path)
testing.expect(t, save_config(cfg, force = true), "save should succeed")
delete_config(&cfg)
}
db, ok := db_open(cfg_path)
testing.expect(t, ok, "db should open")
db_close(&db)
}
@(test)
test_open_existing_db_has_no_leaks :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-leak-existing-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
cfg_path, err := filepath.join([]string{base, "config.json"}, context.temp_allocator)
testing.expect(t, err == nil, "cfgPath should build successfully")
{
cfg := new_config([]string{"fixtures/keys/insecure-test-key"}, cfg_path)
testing.expect(t, save_config(cfg, force = true), "save should succeed")
delete_config(&cfg)
}
// First open/close creates data.envr on disk
db, ok := db_open(cfg_path)
testing.expect(t, ok, "db should open")
if !ok do return
f := make_test_env_file(
"/project/.env",
"abc123",
"SECRET=value",
[]string{"git@github.com:user/repo.git"},
)
defer delete(f.Remotes)
testing.expect(t, db_insert(&db, f), "insert should succeed")
db_close(&db)
// Second open exercises db_restore_from_encrypted
db2, ok2 := db_open(cfg_path)
testing.expect(t, ok2, "db should open existing")
if !ok2 do return
db_close(&db2)
}
@(test)
test_db_sync_noop :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-sync-noop-%d", os.get_pid())
test_env_file_backup :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-backup-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
env_path := fmt.tprintf("%s/.env", base)
content := "KEY=value\n"
write_err := os.write_entire_file(env_path, transmute([]u8)content)
testing.expect(t, write_err == nil, "should write .env file")
err := os.write_entire_file(env_path, "KEY=12345\n")
testing.expect(t, err == nil, ".env file should exist")
digest := hash.hash_bytes(
hash.Algorithm.SHA256,
transmute([]u8)content,
context.temp_allocator,
)
hex_bytes, _ := hex.encode(digest, context.temp_allocator)
sha := string(hex_bytes)
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
defer db_close(&d)
f := make_test_env_file(env_path, sha, content)
f.Dir = base
db_insert(&d, f)
result, sync_err := db_sync(&d, &f)
testing.expect(t, sync_err == .None, "sync should not error")
testing.expect(t, result == {}, "should be noop")
}
@(test)
test_db_sync_backed_up :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-sync-backup-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
env_path := fmt.tprintf("%s/.env", base)
changed_content := "KEY=changed\n"
write_err := os.write_entire_file(env_path, transmute([]u8)changed_content)
testing.expect(t, write_err == nil, "should write .env file")
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
defer db_close(&d)
f := make_test_env_file(env_path, "old_sha", "KEY=original")
f.Dir = base
db_insert(&d, f)
result, sync_err := db_sync(&d, &f)
testing.expect(t, sync_err == .None, "sync should not error")
testing.expect(t, .BackedUp in result, "should be backed up")
}
@(test)
test_db_sync_restored :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-sync-restore-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
env_path := fmt.tprintf("%s/.env", base)
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
defer db_close(&d)
f := make_test_env_file(env_path, "some_sha", "SECRET=value")
f.Dir = base
defer delete(f.Remotes)
db_insert(&d, f)
result, err := db_sync(&d, &f)
testing.expect(t, err == .None, "sync should not error")
testing.expect(t, .Restored in result, "should be restored")
data, read_err := os.read_entire_file_from_path(env_path, context.temp_allocator)
testing.expect(t, read_err == nil, "file should exist after restore")
if read_err == nil {
testing.expect_value(t, string(data), "SECRET=value")
f := EnvFile {
Path = env_path,
}
defer delete(f.contents)
defer delete(f.Sha256)
testing.expect(t, env_file_backup(&f), "backup should succeed")
testing.expect_value(t, f.contents, "KEY=12345\n")
testing.expect_value(t, len(f.Sha256), 64)
}
@(test)
test_db_sync_dir_missing :: proc(t: ^testing.T) {
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
defer db_close(&d)
f := make_test_env_file("/nonexistent/path/.env", "sha", "KEY=val")
db_insert(&d, f)
result, err := db_sync(&d, &f)
testing.expect_value(t, err, SyncError.DirMissing)
testing.expect_value(t, result, nil)
}
@(test)
test_db_sync_moved :: proc(t: ^testing.T) {
base := fmt.tprintf("/tmp/envr-test-sync-moved-%d", os.get_pid())
search_root := fmt.tprintf("%s/search", base)
repo_dir := fmt.tprintf("%s/myproject", search_root)
git_dir := fmt.tprintf("%s/.git", repo_dir)
defer os.remove_all(base)
os.mkdir_all(git_dir)
config_content := "[remote \"origin\"]\n\turl = git@github.com:user/repo.git\n"
config_path := fmt.tprintf("%s/config", git_dir)
write_err := os.write_entire_file(config_path, transmute([]u8)config_content)
testing.expect(t, write_err == nil, "should write .git/config")
d, ok := db_init()
testing.expect(t, ok, "failed to create test db")
defer db_close(&d)
d.cfg.ScanConfig.Include = make([dynamic]string, 0, 1, context.temp_allocator)
append(&d.cfg.ScanConfig.Include, search_root)
f := make_test_env_file(
"/old/nonexistent/path/.env",
"some_sha",
"SECRET=value",
[]string{"git@github.com:user/repo.git"},
)
testing.expect(t, db_insert(&d, f), "insert should succeed")
result, err := db_sync(&d, &f)
testing.expect(t, err == .None, "sync should not error")
if err != .None do return
testing.expect(t, .DirUpdated in result, "should have DirUpdated flag")
testing.expect(t, .Restored in result, "should have Restored flag")
expected_path := fmt.tprintf("%s/.env", repo_dir)
testing.expect_value(t, f.Path, expected_path)
testing.expect_value(t, f.Dir, repo_dir)
_, old_exists := db_fetch(&d, "/old/nonexistent/path/.env")
testing.expect(t, !old_exists, "old path should be deleted from db")
new_fetched, new_ok := db_fetch(&d, expected_path)
testing.expect(t, new_ok, "new path should exist in db")
if new_ok {
testing.expect_value(t, new_fetched.contents, "SECRET=value")
test_env_file_backup_missing :: proc(t: ^testing.T) {
f := EnvFile {
Path = "/tmp/envr-nonexistent-backup/.env",
}
testing.expect(t, !env_file_backup(&f), "missing file should return false")
}
@(test)
test_update_dir :: proc(t: ^testing.T) {
f := EnvFile {
Path = "/old/project/.env",
Dir = "/old/project",
Remotes = make([dynamic]string, 0),
}
defer delete_envfile(&f)
update_dir(&f, "/new/location")
testing.expect_value(t, f.Dir, "/new/location")
testing.expect_value(t, f.Path, "/new/location/.env")
}

View File

@@ -4,11 +4,11 @@ Manage your .env files.
### Synopsis
envr keeps your .env synced to a local, encrypted database.
envr keeps your .env synced to a local, age encrypted database.
Is a safe and eay way to gather all your .env files in one place where they can
easily be backed by another tool such as restic or git.
All your data is stored in ~/.envr/data.envr
All your data is stored in ~/data.age
Getting started is easy:
@@ -45,6 +45,7 @@ at before, restore your backup with:
* [envr backup](envr_backup.md) - Import a .env file into envr
* [envr check](envr_check.md) - check if files in the current directory are backed up
* [envr deps](envr_deps.md) - Check for missing binaries
* [envr edit-config](envr_edit-config.md) - Edit your config with your default editor
* [envr init](envr_init.md) - Set up envr
* [envr list](envr_list.md) - View your tracked files

24
docs/cli/envr_deps.md Normal file
View File

@@ -0,0 +1,24 @@
## envr deps
Check for missing binaries
### Synopsis
envr relies on external binaries for certain functionality.
The check command reports on which binaries are available and which are not.
```
envr deps [flags]
```
### Options
```
-h, --help help for deps
```
### SEE ALSO
* [envr](envr.md) - Manage your .env files.

51
features.odin Normal file
View File

@@ -0,0 +1,51 @@
package main
import "base:runtime"
import "core:mem"
import "core:os"
import "core:strings"
Feature :: enum {
Git,
Fd,
}
AvailableFeatures :: bit_set[Feature]
check_features :: proc() -> AvailableFeatures {
feats: AvailableFeatures
s: mem.Scratch
mem.scratch_init(&s, 4 * mem.DEFAULT_PAGE_SIZE)
defer mem.scratch_destroy(&s)
context.temp_allocator = mem.scratch_allocator(&s)
path_env := os.get_env("PATH", context.temp_allocator)
paths := strings.split(path_env, ":", context.temp_allocator)
if find_binary(paths, "git") != "" {
feats += {.Git}
}
if find_binary(paths, "fd") != "" {
feats += {.Fd}
}
return feats
}
find_binary :: proc(
paths: []string,
name: string,
allocator: runtime.Allocator = context.temp_allocator,
) -> string {
for p in paths {
candidate := strings.join({strings.trim_right(p, "/"), name}, "/", allocator)
_, err := os.stat(candidate, allocator)
if err == nil {
return candidate
}
}
return ""
}

34
features_test.odin Normal file
View File

@@ -0,0 +1,34 @@
package main
import "core:os"
import "core:strings"
import "core:testing"
@(test)
test_find_binary_exists :: proc(t: ^testing.T) {
path := os.get_env("PATH", context.temp_allocator)
paths := strings.split(path, ":", context.temp_allocator)
result := find_binary(paths, "sh")
testing.expect(t, result != "", "sh should be found on PATH")
}
@(test)
test_find_binary_not_exists :: proc(t: ^testing.T) {
old_path := os.get_env("PATH", context.temp_allocator)
defer {
if old_path != "" {
os.set_env("PATH", old_path)
}
}
os.set_env("PATH", "/tmp/envr-nope")
path := os.get_env("PATH", context.temp_allocator)
paths := strings.split(path, ":", context.temp_allocator)
result := find_binary(paths, "no_such_binary_xyz")
testing.expect(t, result == "", "nonexistent binary should not be found")
}

View File

@@ -1,320 +0,0 @@
package findr
import "core:os"
import "core:sort"
import "core:strings"
import "core:sys/linux"
import "core:testing"
// ============================================================================
// Gitignored file emission tests (emit ONLY gitignored files, descend everywhere)
// ============================================================================
@(test)
test_basic_gitignored :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n")
create_file(env, "repo/.env")
create_file(env, "repo/secrets.env")
create_file(env, "repo/normal.txt")
assert_output(t, env, nil, {}, {
"repo/.env", "repo/secrets.env",
})
}
@(test)
test_non_repo_not_scanned :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_dir(env, "norepo")
create_file(env, "norepo/.gitignore", "*.env\n")
create_file(env, "norepo/.env")
assert_output_empty(t, env, nil, {})
}
@(test)
test_negation_pattern :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n!prod.env\n")
create_file(env, "repo/.env")
create_file(env, "repo/secrets.env")
create_file(env, "repo/prod.env")
assert_output(t, env, nil, {}, {
"repo/.env", "repo/secrets.env",
})
}
@(test)
test_multiple_repos :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo1")
create_file(env, "repo1/.gitignore", "*.env\n")
create_file(env, "repo1/a.env")
create_git_repo(env, "repo2")
create_file(env, "repo2/.gitignore", "*.key\n")
create_file(env, "repo2/secret.key")
assert_output(t, env, nil, {}, {
"repo1/a.env", "repo2/secret.key",
})
}
@(test)
test_nested_repos :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "parent")
create_file(env, "parent/.gitignore", "*.env\n")
create_file(env, "parent/top.env")
create_git_repo(env, "parent/child")
create_file(env, "parent/child/.gitignore", "*.key\n")
create_file(env, "parent/child/api.key")
assert_output(t, env, nil, {}, {
"parent/top.env", "parent/child/api.key",
})
}
@(test)
test_nested_gitignore_read :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n")
create_dir(env, "repo/sub")
create_file(env, "repo/sub/.gitignore", "*.txt\n")
create_file(env, "repo/sub/secret.txt")
create_file(env, "repo/sub/.env")
assert_output(t, env, nil, {}, {
"repo/sub/secret.txt", "repo/sub/.env",
})
}
@(test)
test_nested_gitignore_negation :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.log\n")
create_dir(env, "repo/sub")
create_file(env, "repo/sub/.gitignore", "!important.log\n")
create_file(env, "repo/sub/important.log")
create_file(env, "repo/sub/debug.log")
assert_output(t, env, nil, {}, {
"repo/sub/debug.log",
})
}
@(test)
test_multisegment_pattern :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "build/output.txt\n")
create_dir(env, "repo/build")
create_file(env, "repo/build/output.txt")
create_file(env, "repo/build/other.txt")
create_file(env, "repo/output.txt")
assert_output(t, env, nil, {}, {
"repo/build/output.txt",
})
}
@(test)
test_no_gitignore_file :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.env")
assert_output_empty(t, env, nil, {})
}
@(test)
test_empty_gitignore :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "\n\n# comment\n\n")
create_file(env, "repo/.env")
assert_output_empty(t, env, nil, {})
}
@(test)
test_multiple_search_dirs :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "dir1/repo")
create_file(env, "dir1/repo/.gitignore", "*.env\n")
create_file(env, "dir1/repo/a.env")
create_file(env, "dir1/repo/normal.txt")
create_git_repo(env, "dir2/repo")
create_file(env, "dir2/repo/.gitignore", "*.env\n")
create_file(env, "dir2/repo/b.env")
dir1 := join_path(env.temp_dir, "dir1")
defer delete(dir1)
dir2 := join_path(env.temp_dir, "dir2")
defer delete(dir2)
results := make([dynamic]string)
defer {
for r in results {delete(r)}
delete(results)
}
opts := WalkOptions{}
thread_count := os.get_processor_core_count()
walk({dir1, dir2}, &results, opts, thread_count)
testing.expect_value(t, len(results), 2)
actual := make([dynamic]string, 0, len(results))
for r in results {
stripped := r
if strings.has_prefix(stripped, env.temp_dir) {
stripped = stripped[len(env.temp_dir):]
if len(stripped) > 0 && stripped[0] == '/' {
stripped = stripped[1:]
}
}
append(&actual, stripped)
}
defer delete(actual)
expected := []string{"dir1/repo/a.env", "dir2/repo/b.env"}
sort.quick_sort(actual[:])
sort.quick_sort(expected[:])
for i in 0 ..< len(expected) {
testing.expect_value(t, actual[i], expected[i])
}
}
// ============================================================================
// Ignored directory recursion tests
// ============================================================================
@(test)
test_ignored_dir_descended :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "secrets/\n")
create_dir(env, "repo/secrets")
create_file(env, "repo/secrets/.env")
create_file(env, "repo/secrets/api.key")
// Ignored dir's contents are emitted AND descended into
assert_output(t, env, nil, {}, {
"repo/secrets/", "repo/secrets/.env", "repo/secrets/api.key",
})
}
@(test)
test_nested_ignored_dir :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "build/\n")
create_dir(env, "repo/build")
create_dir(env, "repo/build/sub")
create_file(env, "repo/build/output.txt")
create_file(env, "repo/build/sub/deep.env")
assert_output(t, env, nil, {}, {
"repo/build/", "repo/build/output.txt",
"repo/build/sub/", "repo/build/sub/deep.env",
})
}
// ============================================================================
// Filter tests (excludes, pattern)
// ============================================================================
@(test)
test_excludes_prune_dirs :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n")
create_file(env, "repo/.env")
create_dir(env, "repo/vendor")
create_file(env, "repo/vendor/lib.env")
assert_output(t, env, nil,
{excludes = {"vendor"}},
{"repo/.env"},
)
}
@(test)
test_pattern_filters_results :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n*.key\n")
create_file(env, "repo/.env")
create_file(env, "repo/secrets.env")
create_file(env, "repo/master.key")
assert_output(t, env, nil,
{pattern = "\\.env$"},
{"repo/.env", "repo/secrets.env"},
)
}
// ============================================================================
// Special file type tests
// ============================================================================
@(test)
test_fifo_emitted :: proc(t: ^testing.T) {
env := create_test_env()
defer destroy_test_env(&env)
create_git_repo(env, "repo")
create_file(env, "repo/.gitignore", "*.env\n*.fifo\n")
fifo_path := join_path(env.temp_dir, "repo/test.fifo")
defer delete(fifo_path)
cpath := strings.clone_to_cstring(fifo_path)
defer delete(cpath)
linux.mknod(cpath, linux.S_IFIFO | linux.Mode{.IRUSR, .IWUSR}, 0)
assert_output(t, env, nil,
{pattern = "\\.fifo$"},
{"repo/test.fifo"},
)
}

View File

@@ -1,88 +0,0 @@
package findr
import "core:strings"
Gitignore :: struct {
rules: [dynamic]Rule,
}
Rule :: struct {
pattern: GlobPattern,
negated: bool,
dir_only: bool,
}
Match :: enum {
None,
Ignored,
Unignored,
}
is_ignored :: proc(gi: ^Gitignore, path: string, is_dir: bool) -> bool {
return check_match(gi, path, is_dir) == .Ignored
}
check_match :: proc(gi: ^Gitignore, path: string, is_dir: bool) -> Match {
result := Match.None
for &rule in gi.rules {
if rule.dir_only && !is_dir do continue
if glob_match_compiled(&rule.pattern, path) {
result = rule.negated ? .Unignored : .Ignored
}
}
return result
}
parse :: proc(content: string) -> Gitignore {
gi: Gitignore
gi.rules = make([dynamic]Rule)
remaining := content
for {
line, ok := strings.split_lines_iterator(&remaining)
if !ok do break
s := strings.trim_space(line)
if len(s) == 0 do continue
if s[0] == '#' do continue
negated := false
if s[0] == '!' {
negated = true
s = s[1:]
}
if len(s) > 0 && s[0] == '\\' {
if len(s) > 1 && (s[1] == '#' || s[1] == '!') {
s = s[1:]
}
}
dir_only := false
if len(s) > 0 && s[len(s) - 1] == '/' {
dir_only = true
s = s[:len(s) - 1]
}
anchored := false
if len(s) > 0 && s[0] == '/' {
anchored = true
s = s[1:]
}
if len(s) == 0 do continue
gp := glob_compile(s, anchored)
append(&gi.rules, Rule{pattern = gp, negated = negated, dir_only = dir_only})
}
return gi
}
destroy :: proc(gi: ^Gitignore) {
for &rule in gi.rules {
glob_destroy(&rule.pattern)
}
delete(gi.rules)
}

View File

@@ -1,118 +0,0 @@
package findr
import "core:testing"
@(test)
test_is_ignored_basic :: proc(t: ^testing.T) {
gi := parse("*.env\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, ".env", false), true)
testing.expect_value(t, is_ignored(&gi, "foo.env", false), true)
testing.expect_value(t, is_ignored(&gi, ".env.local", false), false)
testing.expect_value(t, is_ignored(&gi, "config.yaml", false), false)
}
@(test)
test_is_ignored_negation :: proc(t: ^testing.T) {
gi := parse("*.env\n!.env.production\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, ".env", false), true)
testing.expect_value(t, is_ignored(&gi, ".env.production", false), false)
}
@(test)
test_is_ignored_dir_only :: proc(t: ^testing.T) {
gi := parse("node_modules/\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, "node_modules", true), true)
testing.expect_value(t, is_ignored(&gi, "node_modules", false), false)
}
@(test)
test_is_ignored_anchored :: proc(t: ^testing.T) {
gi := parse("/secret.key\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, "secret.key", false), true)
}
@(test)
test_is_ignored_comments_skipped :: proc(t: ^testing.T) {
gi := parse("# this is a comment\n#another\n*.tmp\n")
defer destroy(&gi)
testing.expect_value(t, len(gi.rules), 1)
testing.expect_value(t, is_ignored(&gi, "file.tmp", false), true)
}
@(test)
test_is_ignored_blank_lines_skipped :: proc(t: ^testing.T) {
gi := parse("\n\n \n*.log\n\n")
defer destroy(&gi)
testing.expect_value(t, len(gi.rules), 1)
}
@(test)
test_is_ignored_last_match_wins :: proc(t: ^testing.T) {
gi := parse("*.env\n!*.env\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, ".env", false), false)
}
@(test)
test_is_ignored_no_rules :: proc(t: ^testing.T) {
gi := parse("")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, "anything", false), false)
}
@(test)
test_is_ignored_env_pattern :: proc(t: ^testing.T) {
gi := parse(".env*\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, ".env", false), true)
testing.expect_value(t, is_ignored(&gi, ".env.local", false), true)
testing.expect_value(t, is_ignored(&gi, ".envrc", false), true)
}
@(test)
test_is_ignored_globstar :: proc(t: ^testing.T) {
gi := parse("**/cache\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, "cache", false), true)
testing.expect_value(t, is_ignored(&gi, "foo/cache", false), true)
testing.expect_value(t, is_ignored(&gi, "foo/bar/cache", false), true)
}
@(test)
test_star_negation_subpath :: proc(t: ^testing.T) {
gi := parse("*\n!public/\n")
defer destroy(&gi)
// public dir itself is un-ignored
testing.expect_value(t, is_ignored(&gi, "public", true), false)
// children of public/ should still be ignored by *
testing.expect_value(t, is_ignored(&gi, "public/uuid-dir", true), true)
testing.expect_value(t, is_ignored(&gi, "public/uuid-dir/file.txt", false), true)
}
@(test)
test_is_ignored_hash_pattern :: proc(t: ^testing.T) {
gi := parse("\\#*\\#\n")
defer destroy(&gi)
testing.expect_value(t, is_ignored(&gi, "#foo#", false), true)
testing.expect_value(t, is_ignored(&gi, "#test#", false), true)
testing.expect_value(t, is_ignored(&gi, "AUTHORS", false), false)
testing.expect_value(t, is_ignored(&gi, "build.zig", false), false)
testing.expect_value(t, is_ignored(&gi, "ChangeLog", false), false)
}

View File

@@ -1,203 +0,0 @@
package findr
Range :: struct {
lo: u8,
hi: u8,
}
Class_Data :: struct {
negated: bool,
ranges: [dynamic]Range,
}
Token_Kind :: enum u8 { Char, Star, Globstar, Question, Class }
Token :: struct {
kind: Token_Kind,
byte: u8,
class_idx: u16,
}
GlobPattern :: struct {
tokens: [dynamic]Token,
classes: [dynamic]Class_Data,
anchored: bool,
}
glob_compile :: proc(pattern: string, anchored: bool) -> GlobPattern {
gp: GlobPattern
gp.tokens = make([dynamic]Token)
gp.classes = make([dynamic]Class_Data)
gp.anchored = anchored
i := 0
for i < len(pattern) {
c := pattern[i]
if c == '*' {
if i + 1 < len(pattern) && pattern[i + 1] == '*' {
prev_slash := i == 0 || pattern[i - 1] == '/'
at_end := i + 2 >= len(pattern)
next_slash := !at_end && pattern[i + 2] == '/'
if prev_slash && (next_slash || at_end) {
append(&gp.tokens, Token{kind = .Globstar})
if next_slash {
i += 3
} else {
i += 2
}
} else {
append(&gp.tokens, Token{kind = .Star})
i += 2
}
} else {
append(&gp.tokens, Token{kind = .Star})
i += 1
}
} else if c == '?' {
append(&gp.tokens, Token{kind = .Question})
i += 1
} else if c == '[' {
i += 1
negated := false
if i < len(pattern) && pattern[i] == '!' {
negated = true
i += 1
}
ranges := make([dynamic]Range)
if i < len(pattern) && pattern[i] == ']' {
append(&ranges, Range{lo = ']', hi = ']'})
i += 1
}
for i < len(pattern) && pattern[i] != ']' {
if i + 2 < len(pattern) && pattern[i + 1] == '-' && pattern[i + 2] != ']' {
append(&ranges, Range{lo = pattern[i], hi = pattern[i + 2]})
i += 3
} else {
append(&ranges, Range{lo = pattern[i], hi = pattern[i]})
i += 1
}
}
if i < len(pattern) {
i += 1
}
class_idx := u16(len(gp.classes))
append(&gp.classes, Class_Data{negated = negated, ranges = ranges})
append(&gp.tokens, Token{kind = .Class, class_idx = class_idx})
} else if c == '\\' {
i += 1
if i < len(pattern) {
append(&gp.tokens, Token{kind = .Char, byte = pattern[i]})
i += 1
}
} else {
append(&gp.tokens, Token{kind = .Char, byte = c})
i += 1
}
}
return gp
}
match_tokens :: proc(tokens: []Token, classes: []Class_Data, ti: int, path: string, pi: int) -> bool {
if ti >= len(tokens) {
return pi == len(path)
}
tok := tokens[ti]
switch tok.kind {
case .Char:
if pi < len(path) && path[pi] == tok.byte {
return match_tokens(tokens, classes, ti + 1, path, pi + 1)
}
return false
case .Question:
if pi < len(path) && path[pi] != '/' {
return match_tokens(tokens, classes, ti + 1, path, pi + 1)
}
return false
case .Star:
max_end := pi
for max_end < len(path) && path[max_end] != '/' {
max_end += 1
}
for end := max_end; end >= pi; end -= 1 {
if match_tokens(tokens, classes, ti + 1, path, end) {
return true
}
}
return false
case .Globstar:
if ti + 1 >= len(tokens) {
return true
}
if match_tokens(tokens, classes, ti + 1, path, pi) {
return true
}
for end := pi + 1; end <= len(path); end += 1 {
if path[end - 1] == '/' {
if match_tokens(tokens, classes, ti + 1, path, end) {
return true
}
}
}
return false
case .Class:
if pi >= len(path) {
return false
}
cd := classes[tok.class_idx]
ch := path[pi]
in_range := false
for r in cd.ranges {
if ch >= r.lo && ch <= r.hi {
in_range = true
break
}
}
if in_range != cd.negated {
return match_tokens(tokens, classes, ti + 1, path, pi + 1)
}
return false
}
return false
}
glob_match_compiled :: proc(gp: ^GlobPattern, path: string) -> bool {
tokens := gp.tokens[:]
classes := gp.classes[:]
if gp.anchored {
return match_tokens(tokens, classes, 0, path, 0)
}
if match_tokens(tokens, classes, 0, path, 0) {
return true
}
for i := 1; i < len(path); i += 1 {
if path[i - 1] == '/' {
if match_tokens(tokens, classes, 0, path, i) {
return true
}
}
}
return false
}
glob_destroy :: proc(gp: ^GlobPattern) {
for &cd in gp.classes {
delete(cd.ranges)
}
delete(gp.classes)
delete(gp.tokens)
}

View File

@@ -1,111 +0,0 @@
package findr
import "core:testing"
glob_match :: proc(pattern: string, path: string, anchored: bool) -> bool {
gp := glob_compile(pattern, anchored)
result := glob_match_compiled(&gp, path)
glob_destroy(&gp)
return result
}
@(test)
test_glob_simple :: proc(t: ^testing.T) {
testing.expect(t, glob_match("foo", "foo", false))
testing.expect(t, glob_match("foo", "bar/foo", false))
testing.expect(t, !glob_match("foo", "foobar", false))
testing.expect(t, !glob_match("foo", "foo/bar", false))
}
@(test)
test_glob_anchored :: proc(t: ^testing.T) {
testing.expect(t, glob_match("foo", "foo", true))
testing.expect(t, !glob_match("foo", "bar/foo", true))
testing.expect(t, !glob_match("foo", "foobar", true))
}
@(test)
test_glob_star :: proc(t: ^testing.T) {
testing.expect(t, glob_match("*.log", "test.log", false))
testing.expect(t, glob_match("*.log", ".log", false))
testing.expect(t, !glob_match("*.log", "test.txt", false))
testing.expect(t, !glob_match("*.log", "dir/test", false))
}
@(test)
test_glob_question :: proc(t: ^testing.T) {
testing.expect(t, glob_match("?.log", "a.log", false))
testing.expect(t, !glob_match("?.log", "ab.log", false))
testing.expect(t, !glob_match("?.log", ".log", false))
}
@(test)
test_glob_char_class :: proc(t: ^testing.T) {
testing.expect(t, glob_match("[abc].log", "a.log", false))
testing.expect(t, glob_match("[abc].log", "b.log", false))
testing.expect(t, !glob_match("[abc].log", "d.log", false))
}
@(test)
test_glob_negated_class :: proc(t: ^testing.T) {
testing.expect(t, glob_match("[!abc].log", "d.log", false))
testing.expect(t, !glob_match("[!abc].log", "a.log", false))
}
@(test)
test_glob_dot_literal :: proc(t: ^testing.T) {
testing.expect(t, glob_match(".env", ".env", false))
testing.expect(t, glob_match(".env", "dir/.env", false))
testing.expect(t, !glob_match(".env", "env", false))
testing.expect(t, !glob_match(".env", "x.env", false))
}
@(test)
test_glob_globstar_prefix :: proc(t: ^testing.T) {
testing.expect(t, glob_match("**/foo", "foo", false))
testing.expect(t, glob_match("**/foo", "a/b/foo", false))
testing.expect(t, !glob_match("**/foo", "foobar", false))
testing.expect(t, !glob_match("**/foo", "a/foobar", false))
}
@(test)
test_glob_globstar_suffix :: proc(t: ^testing.T) {
testing.expect(t, glob_match("abc/**", "abc/x", false))
testing.expect(t, glob_match("abc/**", "abc/x/y", false))
testing.expect(t, !glob_match("abc/**", "abc", false))
testing.expect(t, !glob_match("abc/**", "abcd/x", false))
}
@(test)
test_glob_globstar_middle :: proc(t: ^testing.T) {
testing.expect(t, glob_match("foo/**/bar", "foo/bar", false))
testing.expect(t, glob_match("foo/**/bar", "foo/x/bar", false))
testing.expect(t, !glob_match("foo/**/bar", "foo/barx", false))
testing.expect(t, !glob_match("foo/**/bar", "foo/x/y/baz", false))
}
@(test)
test_glob_backslash_escape :: proc(t: ^testing.T) {
testing.expect(t, glob_match("\\!foo", "!foo", false))
testing.expect(t, !glob_match("\\!foo", "foo", false))
}
@(test)
test_glob_hash_literal :: proc(t: ^testing.T) {
testing.expect(t, glob_match("#foo", "#foo", false))
testing.expect(t, !glob_match("#foo", "foo", false))
}
@(test)
test_glob_hash_pattern :: proc(t: ^testing.T) {
testing.expect(t, glob_match("#*#", "#test#", false))
testing.expect(t, glob_match("#*#", "##", false))
testing.expect(t, !glob_match("#*#", "test", false))
testing.expect(t, !glob_match("#*#", "#test", false))
}
@(test)
test_glob_empty :: proc(t: ^testing.T) {
testing.expect(t, glob_match("", "", false))
testing.expect(t, !glob_match("", "foo", false))
}

View File

@@ -1,128 +0,0 @@
package findr
import "core:strings"
import "core:sync"
import "core:sys/linux"
import "core:thread"
RepoPool :: struct {
queue: [dynamic]string,
queue_mutex: sync.Mutex,
queue_sema: sync.Atomic_Sema,
results: ^[dynamic]string,
results_lock: sync.Mutex,
active: i64,
done: sync.One_Shot_Event,
threads: []^thread.Thread,
}
find_repos :: proc(roots: []string, results: ^[dynamic]string, thread_count: int) {
if len(roots) == 0 do return
pool := new(RepoPool)
pool.queue = make([dynamic]string)
pool.results = results
pool.active = i64(len(roots))
pool.threads = make([]^thread.Thread, thread_count)
for root in roots {
root_clone, _ := strings.clone(root)
append(&pool.queue, root_clone)
sync.atomic_sema_post(&pool.queue_sema)
}
for i in 0 ..< thread_count {
t := thread.create(repo_worker)
t.data = rawptr(pool)
t.init_context = context
thread.start(t)
pool.threads[i] = t
}
sync.one_shot_event_wait(&pool.done)
for _ in 0 ..< thread_count {
sync.atomic_sema_post(&pool.queue_sema)
}
for t in pool.threads {
thread.destroy(t)
}
delete(pool.threads)
for path in pool.queue {
delete(path)
}
delete(pool.queue)
free(pool)
}
repo_worker :: proc(t: ^thread.Thread) {
pool := cast(^RepoPool)t.data
for {
sync.atomic_sema_wait(&pool.queue_sema)
sync.mutex_lock(&pool.queue_mutex)
if len(pool.queue) == 0 {
sync.mutex_unlock(&pool.queue_mutex)
if sync.atomic_load_explicit(&pool.active, .Acquire) == 0 {
sync.one_shot_event_signal(&pool.done)
}
break
}
last := len(pool.queue) - 1
dir_path := pool.queue[last]
ordered_remove(&pool.queue, last)
sync.mutex_unlock(&pool.queue_mutex)
process_repo_dir(pool, dir_path)
delete(dir_path)
old := sync.atomic_sub_explicit(&pool.active, 1, .Release)
if old == 1 {
sync.one_shot_event_signal(&pool.done)
}
}
}
process_repo_dir :: proc(pool: ^RepoPool, dir_path: string) {
cpath := strings.clone_to_cstring(dir_path)
if cpath == nil do return
defer delete(cpath)
fd, open_err := linux.open(cpath, {.DIRECTORY, .CLOEXEC})
if open_err != .NONE do return
defer linux.close(fd)
if has_git_dir(fd) {
cloned, _ := strings.clone(dir_path)
sync.mutex_lock(&pool.results_lock)
append(pool.results, cloned)
sync.mutex_unlock(&pool.results_lock)
}
buf: [32 * 1024]u8
for {
n, errno := linux.getdents(fd, buf[:])
if n <= 0 || errno != .NONE do break
offs := 0
for d in linux.dirent_iterate_buf(buf[:n], &offs) {
name := linux.dirent_name(d)
if name == "." || name == ".." do continue
if name == ".git" do continue
if d.type == .DIR {
child_path := join_path(dir_path, name)
sync.atomic_add_explicit(&pool.active, 1, .Relaxed)
sync.mutex_lock(&pool.queue_mutex)
append(&pool.queue, child_path)
sync.mutex_unlock(&pool.queue_mutex)
sync.atomic_sema_post(&pool.queue_sema)
}
}
}
}

View File

@@ -1,152 +0,0 @@
package findr
import "core:fmt"
import "core:log"
import "core:os"
import "core:sort"
import "core:strings"
import "core:testing"
TestEnv :: struct {
temp_dir: string,
}
create_test_env :: proc() -> (env: TestEnv) {
tmp, err := os.mkdir_temp("", "findr-test-*", context.allocator)
if err != nil {
log.error("Failed to create temp dir:", err)
panic("Failed to create temp dir")
}
env.temp_dir = tmp
return
}
destroy_test_env :: proc(env: ^TestEnv) {
os.remove_all(env.temp_dir)
delete(env.temp_dir)
}
create_dir :: proc(env: TestEnv, path: string) {
full := join_path(env.temp_dir, path)
defer delete(full)
os.mkdir_all(full, os.Permissions_Default_Directory)
}
create_file :: proc(env: TestEnv, path: string, content: string = "") {
full := join_path(env.temp_dir, path)
defer delete(full)
dir_end := strings.last_index(full, "/")
if dir_end >= 0 {
dir_path := full[:dir_end]
os.mkdir_all(dir_path, os.Permissions_Default_Directory)
}
f, err := os.create(full)
if err != nil {
log.error("Failed to create file:", full, err)
return
}
if len(content) > 0 {
os.write_string(f, content)
}
os.close(f)
}
create_git_repo :: proc(env: TestEnv, path: string) {
sub := join_path(path, ".git")
defer delete(sub)
create_dir(env, sub)
}
assert_output :: proc(
t: ^testing.T,
env: TestEnv,
args: []string,
opts: WalkOptions,
expected: []string,
) {
results := collect_results(env, args, opts)
defer {
for r in results {delete(r)}
delete(results)
}
sorted_expected := make([dynamic]string, 0, len(expected))
for e in expected {append(&sorted_expected, e)}
defer delete(sorted_expected)
sorted_actual := make([dynamic]string, 0, len(results))
for a in results {append(&sorted_actual, a)}
defer delete(sorted_actual)
sort.quick_sort(sorted_expected[:])
sort.quick_sort(sorted_actual[:])
if len(sorted_expected) != len(sorted_actual) {
testing.fail(t)
log.error(
fmt.tprintf("Expected %d results, got %d", len(sorted_expected), len(sorted_actual)),
)
log.error("Expected:", sorted_expected[:])
log.error("Actual: ", sorted_actual[:])
return
}
for i in 0 ..< len(sorted_expected) {
if sorted_expected[i] != sorted_actual[i] {
testing.fail(t)
log.error(fmt.tprintf("Mismatch at index %d", i))
log.error("Expected:", sorted_expected[:])
log.error("Actual: ", sorted_actual[:])
return
}
}
}
assert_output_empty :: proc(
t: ^testing.T,
env: TestEnv,
args: []string,
opts: WalkOptions,
) {
results := collect_results(env, args, opts)
defer {
for r in results {delete(r)}
delete(results)
}
if len(results) > 0 {
testing.fail(t)
log.error(fmt.tprintf("Expected no results, got %d:", len(results)))
for r in results {
log.error(" ", r)
}
}
}
collect_results :: proc(env: TestEnv, args: []string, opts: WalkOptions) -> [dynamic]string {
results := make([dynamic]string)
full_args := make([dynamic]string, 0, len(args) + 1, context.temp_allocator)
append(&full_args, env.temp_dir)
for a in args {append(&full_args, a)}
thread_count := os.get_processor_core_count()
walk(full_args[:], &results, opts, thread_count)
for i in 0 ..< len(results) {
r := results[i]
if strings.has_prefix(r, env.temp_dir) {
stripped := r[len(env.temp_dir):]
if len(stripped) > 0 && stripped[0] == '/' {
stripped = stripped[1:]
}
new_r, _ := strings.clone(stripped)
delete(r)
results[i] = new_r
}
}
return results
}

View File

@@ -1,458 +0,0 @@
package findr
import "core:bytes"
import "core:fmt"
import "core:os"
import "core:strings"
import "core:sync"
import "core:sync/chan"
import "core:sys/linux"
import "core:text/regex"
import "core:thread"
OUTPUT_BUF_SIZE :: 64 * 1024
WalkOptions :: struct {
pattern: string, // regex on basename; "" = match all
excludes: []string, // glob patterns to skip entirely
}
GIContext :: struct {
gi: ^Gitignore, // nil if this dir had no .gitignore
base_rel: string, // relative path from repo root to this dir
parent: ^GIContext, // parent context (nil if repo root)
}
WorkItem :: struct {
path: string, // absolute directory path
rel: string, // relative path from repo root ("" = root)
gi_ctx: ^GIContext, // gitignore chain (nil = outside any repo)
in_repo: bool, // true if inside a git repo
in_ignored: bool, // true if inside a gitignored directory
}
WalkerPool :: struct {
queue: [dynamic]WorkItem,
queue_mutex: sync.Mutex,
queue_sema: sync.Atomic_Sema,
result_chan: chan.Chan([]u8),
active: i64,
done: sync.One_Shot_Event,
threads: []^thread.Thread,
opts: WalkOptions,
pattern_re: regex.Regular_Expression,
has_pattern: bool,
exclude_gi: ^Gitignore,
all_contexts: [dynamic]^GIContext,
contexts_lock: sync.Mutex,
}
Collector_Data :: struct {
ch: chan.Chan([]u8),
results: ^[dynamic]string,
}
collect_worker :: proc(t: ^thread.Thread) {
data := cast(^Collector_Data)t.data
for {
batch := chan.recv(data.ch) or_break
defer delete(batch)
start := 0
for {
remaining: []u8
#no_bounds_check {remaining = batch[start:]}
idx := bytes.index_byte(remaining, '\n')
if idx < 0 do break
i := start + idx
if i > start {
segment: []u8
#no_bounds_check {segment = batch[start:i]}
s, _ := strings.clone(string(segment))
append(data.results, s)
}
start = i + 1
}
}
}
walk :: proc(roots: []string, results: ^[dynamic]string, opts: WalkOptions, thread_count: int) {
if len(roots) == 0 do return
ch, _ := chan.create(chan.Chan([]u8), max(2 * thread_count, 2), context.allocator)
defer chan.destroy(ch)
data := new(Collector_Data)
data.ch = ch
data.results = results
defer free(data)
collector := thread.create(collect_worker)
collector.data = rawptr(data)
collector.init_context = context
thread.start(collector)
pool := new(WalkerPool)
pool.queue = make([dynamic]WorkItem)
pool.result_chan = ch
pool.active = i64(len(roots))
pool.threads = make([]^thread.Thread, thread_count)
pool.all_contexts = make([dynamic]^GIContext)
pool.opts = opts
pool.exclude_gi = nil
pool.has_pattern = false
if len(opts.pattern) > 0 {
re, err := regex.create(opts.pattern, {regex.Flag.No_Capture})
if err == nil {
pool.pattern_re = re
pool.has_pattern = true
}
}
if len(opts.excludes) > 0 {
sb: strings.Builder
strings.builder_init(&sb)
for ex in opts.excludes {
fmt.sbprintf(&sb, "%s\n", ex)
}
content := strings.to_string(sb)
pool.exclude_gi = new(Gitignore)
pool.exclude_gi^ = parse(content)
strings.builder_destroy(&sb)
}
for root in roots {
root_clone, _ := strings.clone(root)
append(&pool.queue, WorkItem{path = root_clone})
sync.atomic_sema_post(&pool.queue_sema)
}
for i in 0 ..< thread_count {
t := thread.create(walk_worker)
t.data = rawptr(pool)
t.init_context = context
thread.start(t)
pool.threads[i] = t
}
sync.one_shot_event_wait(&pool.done)
for _ in 0 ..< thread_count {
sync.atomic_sema_post(&pool.queue_sema)
}
for t in pool.threads {
thread.destroy(t)
}
delete(pool.threads)
for item in pool.queue {
delete(item.path)
if len(item.rel) > 0 {delete(item.rel)}
}
delete(pool.queue)
for ctx in pool.all_contexts {
if ctx.gi != nil {
destroy(ctx.gi)
free(ctx.gi)
}
if len(ctx.base_rel) > 0 {
delete(ctx.base_rel)
}
free(ctx)
}
delete(pool.all_contexts)
if pool.has_pattern {
regex.destroy(pool.pattern_re)
}
if pool.exclude_gi != nil {
destroy(pool.exclude_gi)
free(pool.exclude_gi)
}
free(pool)
chan.close(ch)
thread.join(collector)
thread.destroy(collector)
}
flush_buf :: proc(ch: chan.Chan([]u8), local: ^[dynamic]u8) {
if len(local) == 0 do return
batch := local[:]
local^ = make([dynamic]u8, 0, OUTPUT_BUF_SIZE)
chan.send(ch, batch)
}
append_path :: proc(buf: ^[dynamic]u8, parent, name: string, trailing_slash: bool) {
need_sep := len(parent) > 0 && parent[len(parent) - 1] != '/'
size := len(parent) + len(name) + 1
if need_sep do size += 1
if trailing_slash do size += 1
old_len := len(buf)
reserve(buf, old_len + size)
resize(buf, old_len + size)
pos := old_len
pos += copy(buf[pos:], parent)
if need_sep {buf[pos] = '/'; pos += 1}
pos += copy(buf[pos:], name)
if trailing_slash {buf[pos] = '/'; pos += 1}
buf[pos] = '\n'
}
walk_worker :: proc(t: ^thread.Thread) {
pool := cast(^WalkerPool)t.data
local_buf := make([dynamic]u8, 0, OUTPUT_BUF_SIZE)
defer {
if len(local_buf) > 0 {
flush_buf(pool.result_chan, &local_buf)
}
delete(local_buf)
}
for {
sync.atomic_sema_wait(&pool.queue_sema)
sync.mutex_lock(&pool.queue_mutex)
if len(pool.queue) == 0 {
sync.mutex_unlock(&pool.queue_mutex)
if sync.atomic_load_explicit(&pool.active, .Acquire) == 0 {
sync.one_shot_event_signal(&pool.done)
}
break
}
last := len(pool.queue) - 1
item := pool.queue[last]
ordered_remove(&pool.queue, last)
sync.mutex_unlock(&pool.queue_mutex)
process_dir(pool, item, &local_buf)
delete(item.path)
if len(item.rel) > 0 {delete(item.rel)}
if len(local_buf) >= OUTPUT_BUF_SIZE {
flush_buf(pool.result_chan, &local_buf)
}
old := sync.atomic_sub_explicit(&pool.active, 1, .Release)
if old == 1 {
sync.one_shot_event_signal(&pool.done)
}
}
}
process_dir :: proc(pool: ^WalkerPool, item: WorkItem, local_buf: ^[dynamic]u8) {
dir_path := item.path
cpath := strings.clone_to_cstring(dir_path)
if cpath == nil do return
defer delete(cpath)
fd, open_err := linux.open(cpath, {.DIRECTORY, .CLOEXEC})
if open_err != .NONE do return
defer linux.close(fd)
has_git := false
if !item.in_ignored {
has_git = has_git_dir(fd)
}
gi_ctx := item.gi_ctx
rel := item.rel
if has_git {
gi_ctx = nil
rel = ""
}
child_in_repo := has_git || item.in_repo
gi: ^Gitignore = nil
if !item.in_ignored {
gi = load_ignore_patterns(dir_path, child_in_repo)
}
if gi != nil {
new_ctx := new(GIContext)
new_ctx.gi = gi
if len(rel) > 0 {
new_ctx.base_rel, _ = strings.clone(rel)
}
new_ctx.parent = gi_ctx
sync.mutex_lock(&pool.contexts_lock)
append(&pool.all_contexts, new_ctx)
sync.mutex_unlock(&pool.contexts_lock)
gi_ctx = new_ctx
}
buf: [32 * 1024]u8
rel_buf: [4096]u8
for {
n, errno := linux.getdents(fd, buf[:])
if n <= 0 || errno != .NONE do break
offs := 0
for d in linux.dirent_iterate_buf(buf[:n], &offs) {
name := linux.dirent_name(d)
if name == "." || name == ".." do continue
if name == ".git" do continue
is_dir := d.type == .DIR
is_nondir := d.type != .DIR
if pool.exclude_gi != nil && is_ignored(pool.exclude_gi, name, is_dir) {
continue
}
entry_rel := build_rel(rel_buf[:], rel, name)
ignored := false
if item.in_ignored {
ignored = true
} else if gi_ctx != nil {
ignored = check_chain(gi_ctx, entry_rel, is_dir)
}
if is_dir {
if ignored && matches_pattern(pool, name) {
append_path(local_buf, dir_path, name, true)
}
child_rel, _ := strings.clone(entry_rel)
child_path := join_path(dir_path, name)
push_work(
pool,
WorkItem {
path = child_path,
rel = child_rel,
gi_ctx = gi_ctx,
in_repo = child_in_repo,
in_ignored = ignored,
},
)
} else if is_nondir {
if ignored && matches_pattern(pool, name) {
append_path(local_buf, dir_path, name, false)
}
}
}
}
}
check_chain :: proc(ctx: ^GIContext, entry_rel: string, is_dir: bool) -> bool {
c := ctx
for c != nil {
if c.gi != nil {
rel := relative_to(entry_rel, c.base_rel)
match := check_match(c.gi, rel, is_dir)
if match != .None {
return match == .Ignored
}
}
c = c.parent
}
return false
}
relative_to :: proc(entry_rel, base_rel: string) -> string {
if len(base_rel) == 0 do return entry_rel
prefix_len := len(base_rel)
if len(entry_rel) > prefix_len &&
entry_rel[prefix_len] == '/' &&
strings.has_prefix(entry_rel, base_rel) {
return entry_rel[prefix_len + 1:]
}
return entry_rel
}
build_rel :: proc(buf: []u8, rel, name: string) -> string {
if len(rel) == 0 do return name
pos := copy(buf, rel)
if pos < len(buf) {
buf[pos] = '/'
pos += 1
pos += copy(buf[pos:], name)
}
return string(buf[:pos])
}
matches_pattern :: proc(pool: ^WalkerPool, name: string) -> bool {
if !pool.has_pattern do return true
cap, ok := regex.match(pool.pattern_re, name)
regex.destroy(cap)
return ok
}
push_work :: proc(pool: ^WalkerPool, item: WorkItem) {
sync.atomic_add_explicit(&pool.active, 1, .Relaxed)
sync.mutex_lock(&pool.queue_mutex)
append(&pool.queue, item)
sync.mutex_unlock(&pool.queue_mutex)
sync.atomic_sema_post(&pool.queue_sema)
}
has_git_dir :: proc(fd: linux.Fd) -> bool {
git_fd, err := linux.openat(fd, ".git", {.DIRECTORY, .CLOEXEC})
if err == .NONE {
linux.close(git_fd)
return true
}
return false
}
load_ignore_patterns :: proc(dir_path: string, in_repo: bool) -> ^Gitignore {
has_patterns := false
sb: strings.Builder
strings.builder_init(&sb)
defer strings.builder_destroy(&sb)
if in_repo {
gi_path := join_path(dir_path, ".gitignore")
data, err := os.read_entire_file_from_path(gi_path, context.allocator)
delete(gi_path)
if err == .NONE {
fmt.sbprintf(&sb, "%s", string(data))
delete(data)
has_patterns = true
}
}
ig_path := join_path(dir_path, ".ignore")
idata, ierr := os.read_entire_file_from_path(ig_path, context.allocator)
delete(ig_path)
if ierr == .NONE {
fmt.sbprintf(&sb, "%s", string(idata))
delete(idata)
has_patterns = true
}
if !has_patterns do return nil
content := strings.to_string(sb)
gi := new(Gitignore)
gi^ = parse(content)
return gi
}
join_path :: proc(parent, child: string) -> string {
need_sep := len(parent) == 0 || parent[len(parent) - 1] != '/'
total := len(parent) + len(child)
if need_sep do total += 1
buf := make([]u8, total, context.allocator)
pos := copy(buf, parent)
if need_sep {
buf[pos] = '/'
pos += 1
}
copy(buf[pos:], child)
return string(buf)
}

View File

@@ -66,7 +66,7 @@
packages.default = pkgs.stdenv.mkDerivation rec {
pname = "envr";
version = "0.3.0";
version = "0.2.0";
src = ./.;
nativeBuildInputs = [
@@ -79,13 +79,6 @@
mysqlite
];
doCheck = true;
checkPhase = ''
runHook preCheck
odin test . -all-packages
runHook postCheck
'';
buildPhase = ''
runHook preBuild
echo '${version}' > version.txt
@@ -102,6 +95,7 @@
devShells.default = pkgs.mkShell {
buildInputs = with pkgs; [
fd
nushell
libsodium
@@ -112,10 +106,6 @@
# Build tools
zip
# Helper tools
delta
hyperfine
# IDE
unstable.helix
typescript-language-server

View File

@@ -1,57 +1,14 @@
package main
import "base:runtime"
import "core:bufio"
import "core:fmt"
import "core:mem"
import "core:os"
import "core:prof/spall"
import "core:sync"
SPALL :: #config(SPALL, false)
when SPALL {
spall_ctx: spall.Context
@(thread_local)
spall_buffer: spall.Buffer
}
main :: proc() {
when SPALL {
ctx, spall_ok := spall.context_create_with_scale("envr.spall", false, 1.0)
if !spall_ok {
fmt.eprintln("Failed to create spall trace file")
os.exit(1)
}
spall_ctx = ctx
defer spall.context_destroy(&spall_ctx)
spall_backing := make([]u8, spall.BUFFER_DEFAULT_SIZE)
defer delete(spall_backing)
spall_buffer = spall.buffer_create(spall_backing, u32(sync.current_thread_id()))
defer spall.buffer_destroy(&spall_ctx, &spall_buffer)
}
when ODIN_DEBUG {
heap_track: mem.Tracking_Allocator
mem.tracking_allocator_init(&heap_track, context.allocator)
defer mem.tracking_allocator_destroy(&heap_track)
defer if len(heap_track.allocation_map) > 0 {
for _, leak in heap_track.allocation_map {
fmt.eprintf("LEAK: %v leaked %m\n", leak.location, leak.size)
}
}
context.allocator = mem.tracking_allocator(&heap_track)
temp_track: mem.Tracking_Allocator
mem.tracking_allocator_init(&temp_track, context.temp_allocator)
defer mem.tracking_allocator_destroy(&temp_track)
context.temp_allocator = mem.tracking_allocator(&temp_track)
}
defer free_all(context.temp_allocator)
cmd, ok := parse_args(os.args, os.to_writer(os.stdout), os.to_writer(os.stderr))
defer delete_command(&cmd) // delete flushes automatically
defer bufio.writer_flush(cmd.out_buf)
if !ok {
return
}
@@ -61,6 +18,8 @@ main :: proc() {
cmd_init(&cmd)
case "version":
cmd_version(&cmd)
case "deps":
cmd_deps(&cmd)
case "list":
cmd_list(&cmd)
case "backup", "add":
@@ -86,21 +45,3 @@ main :: proc() {
}
}
when SPALL {
@(instrumentation_enter)
spall_enter :: proc "contextless" (
proc_address, call_site_return_address: rawptr,
loc: runtime.Source_Code_Location,
) {
spall._buffer_begin(&spall_ctx, &spall_buffer, "", "", loc)
}
@(instrumentation_exit)
spall_exit :: proc "contextless" (
proc_address, call_site_return_address: rawptr,
loc: runtime.Source_Code_Location,
) {
spall._buffer_end(&spall_ctx, &spall_buffer)
}
}

View File

@@ -2,7 +2,6 @@ package main
import "core:fmt"
import "core:sys/posix"
import "core:terminal/ansi"
MultiSelect_Result :: enum {
Confirm,
@@ -37,16 +36,16 @@ multi_select :: proc(
return
}
selected = make([dynamic]bool, len(options))
selected = make([dynamic]bool, 0, len(options))
cursor: int = 0
scroll_offset: int = 0
fmt.printf(ansi.CSI + ansi.DECTCEM_HIDE)
fmt.printf("\x1b[?25l")
visible := render_options(prompt, options, selected[:], cursor, scroll_offset)
raw, ok := enable_raw_mode(posix.STDIN_FILENO)
if !ok {
fmt.printf(ansi.CSI + ansi.DECTCEM_SHOW)
fmt.printf("\x1b[?25h")
return
}
defer disable_raw_mode(&raw)
@@ -66,18 +65,18 @@ multi_select :: proc(
case .Space:
selected[cursor] = !selected[cursor]
case .Enter:
fmt.printf(ansi.CSI + "%d" + ansi.CUU + ansi.CSI + ansi.ED + ansi.CSI + ansi.DECTCEM_SHOW, visible + 1)
fmt.printf("\x1b[%dA\x1b[J\x1b[?25h", visible + 1)
result = .Confirm
return
case .Escape:
fmt.printf(ansi.CSI + "%d" + ansi.CUU + ansi.CSI + ansi.ED + ansi.CSI + ansi.DECTCEM_SHOW, visible + 1)
fmt.printf("\x1b[%dA\x1b[J\x1b[?25h", visible + 1)
result = .Cancel
return
case .Unknown:
}
scroll_offset = max(0, min(cursor - MAX_VISIBLE / 2, len(options) - MAX_VISIBLE))
fmt.printf(ansi.CSI + "%d" + ansi.CUU + ansi.CSI + ansi.RESET + ansi.ED, visible + 1)
fmt.printf("\x1b[%dA\x1b[0J", visible + 1)
visible = render_options(prompt, options, selected[:], cursor, scroll_offset)
}
}
@@ -89,7 +88,7 @@ render_options :: proc(
cursor: int,
scroll_offset: int,
) -> int {
fmt.printf(ansi.CSI + ansi.BOLD + ";" + ansi.FG_CYAN + ansi.SGR + "%s" + ANSI_RESET + " (↑/↓ move, space select, enter confirm)\r\n", prompt)
fmt.printf("\x1b[1;36m%s\x1b[0m (↑/↓ move, space select, enter confirm)\r\n", prompt)
end := scroll_offset + MAX_VISIBLE
if end > len(options) {
@@ -102,9 +101,9 @@ render_options :: proc(
checkbox = "x"
}
if i == cursor {
fmt.printf(ansi.CSI + ansi.BOLD + ";" + ansi.FG_GREEN + ansi.SGR + "> " + ANSI_RESET + "[" + ansi.CSI + ansi.FG_GREEN + ansi.SGR + "%s" + ANSI_RESET + "] %s\r\n", checkbox, options[i])
fmt.printf("\x1b[1;32m> \x1b[0m[\x1b[32m%s\x1b[0m] %s\r\n", checkbox, options[i])
} else {
fmt.printf(" [" + ansi.CSI + ansi.FAINT + ansi.SGR + "%s" + ANSI_RESET + "] %s\r\n", checkbox, options[i])
fmt.printf(" [\x1b[2m%s\x1b[0m] %s\r\n", checkbox, options[i])
}
}

127
scan.odin
View File

@@ -1,21 +1,137 @@
package main
import "core:fmt"
import "core:os"
import "core:strings"
import "core:sync"
import "core:terminal"
import "findr"
fd_counter: sync.Atomic_Mutex
fd_seq: int
// Caller is responsible for freeing paths
scan_path :: proc(search_path: string, cfg: Config) -> (paths: [dynamic]string, ok: bool) {
opts := findr.WalkOptions {
pattern = cfg.ScanConfig.Matcher,
excludes = cfg.ScanConfig.Exclude[:],
if terminal.is_terminal(os.stdout) {
fmt.printf("Searching for all files in \"%s\"...\n", search_path)
}
findr.walk({search_path}, &paths, opts, os.get_processor_core_count())
all_files, all_ok := run_fd(build_fd_args(search_path, cfg, true))
if !all_ok {
return
}
if terminal.is_terminal(os.stdout) {
fmt.printf("Search for unignored fies in \"%s\"...\n", search_path)
}
unignored_files, unignored_ok := run_fd(build_fd_args(search_path, cfg, false))
if !unignored_ok {
return
}
unignored_set := make(map[string]bool, len(unignored_files), context.temp_allocator)
for file in unignored_files {
unignored_set[file] = true
}
for file in all_files {
if !(file in unignored_set) {
append(&paths, file)
}
}
ok = true
return
}
@(private = "file")
build_fd_args :: proc(search_path: string, cfg: Config, include_ignored: bool) -> []string {
args_len := 3 + 2 * len(cfg.ScanConfig.Exclude) + 2
args := make([dynamic]string, 0, args_len, context.temp_allocator)
append(&args, "fd")
append(&args, "-a")
append(&args, cfg.ScanConfig.Matcher)
for exclude in cfg.ScanConfig.Exclude {
append(&args, "-E")
append(&args, exclude)
}
if include_ignored {
append(&args, "-HI")
} else {
append(&args, "-H")
}
append(&args, search_path)
return args[:]
}
run_fd :: proc(args: []string) -> (lines: []string, ok: bool) {
tmp_path := next_fd_tmp_path()
tmp_file, tmp_err := os.open(tmp_path, os.O_CREATE | os.O_WRONLY | os.O_TRUNC)
if tmp_err != nil {
// TODO: Log a message here
return
}
desc := os.Process_Desc {
command = args,
stdout = tmp_file,
stderr = nil,
}
p, start_err := os.process_start(desc)
os.close(tmp_file)
if start_err != nil {
os.remove(tmp_path)
return
}
state, wait_err := os.process_wait(p)
if wait_err != nil || state.exit_code != 0 {
os.remove(tmp_path)
return
}
data, read_err := os.read_entire_file_from_path(tmp_path, context.temp_allocator)
os.remove(tmp_path)
if read_err != nil {
return
}
output := string(data)
output = strings.trim_space(output)
if len(output) == 0 {
ok = true
return
}
raw_lines := strings.split(output, "\n", context.temp_allocator)
result := make([dynamic]string, 0, len(raw_lines), context.temp_allocator)
for line in raw_lines {
trimmed := strings.trim_space(line)
if len(trimmed) > 0 {
append(&result, trimmed)
}
}
return result[:], true
}
@(private = "file")
next_fd_tmp_path :: proc() -> string {
sync.atomic_mutex_lock(&fd_counter)
n := fd_seq
fd_seq += 1
sync.atomic_mutex_unlock(&fd_counter)
return fmt.tprintf("/tmp/envr-fd-%d-%d", os.get_pid(), n)
}
cant_scan :: proc(feats: AvailableFeatures) -> bool {
return Feature.Fd not_in feats
}
find_unbacked :: proc(local_files: []string, db_files: []EnvFile) -> []string {
// Lives until the end of the function
backed_set := make(map[string]bool, len(db_files), context.temp_allocator)
for file in db_files {
backed_set[file.Path] = true
@@ -29,3 +145,4 @@ find_unbacked :: proc(local_files: []string, db_files: []EnvFile) -> []string {
}
return unbacked[:]
}

View File

@@ -1,13 +1,16 @@
#+test
package main
import "core:fmt"
import "core:os"
import "core:path/filepath"
import "core:strings"
import "core:testing"
@(test)
test_scan_path_finds_gitignored_env_files :: proc(t: ^testing.T) {
feats := check_features()
testing.expect(t, cant_scan(feats) == false)
base := fmt.tprintf("/tmp/envr-scan-test-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
@@ -39,12 +42,7 @@ test_scan_path_finds_gitignored_env_files :: proc(t: ^testing.T) {
}
results, ok := scan_path(base, cfg)
defer {
for path in results {
delete(path)
}
delete(results)
}
defer delete(results)
testing.expect(t, ok, "scan_path should succeed")
found_env := false
@@ -71,6 +69,9 @@ test_scan_path_finds_gitignored_env_files :: proc(t: ^testing.T) {
@(test)
test_scan_path_empty_dir :: proc(t: ^testing.T) {
feats := check_features()
testing.expect(t, cant_scan(feats) == false)
base := fmt.tprintf("/tmp/envr-scan-empty-%d", os.get_pid())
os.mkdir_all(base)
defer os.remove_all(base)
@@ -84,3 +85,12 @@ test_scan_path_empty_dir :: proc(t: ^testing.T) {
testing.expect(t, ok, "scan_path should succeed")
testing.expect(t, len(results) == 0, fmt.tprintf("expected 0 results, got %d", len(results)))
}
@(test)
test_scan_meets_expectations :: proc(t: ^testing.T) {
testing.expect(t, cant_scan({}), "no features should mean can't scan")
testing.expect(t, cant_scan({.Git}), "Git alone should mean can't scan")
testing.expect(t, !cant_scan({.Fd}), "having Fd should mean can scan")
testing.expect(t, !cant_scan({.Fd, .Git}), "both Fd and Git should mean can scan")
}

View File

@@ -4,51 +4,42 @@ import "core:c"
foreign import lib "system:sqlite3"
Db :: distinct rawptr
Stmt :: distinct rawptr
// TODO: Use an enum?
OK :: 0
ROW :: 100
DONE :: 101
DESERIALIZE_FLAGS :: bit_set[DESERIALIZE_FLAG]
DESERIALIZE_FLAG :: enum u32 {
FREEONCLOSE = 1,
RESIZEABLE = 2,
}
DESERIALIZE_FREEONCLOSE :: 1
DESERIALIZE_RESIZEABLE :: 2
foreign lib {
@(link_name = "sqlite3_open")
open :: proc(filename: cstring, ppDb: ^Db) -> c.int ---
@(link_name = "sqlite3_close")
close :: proc(db: Db) -> c.int ---
@(link_name = "sqlite3_errmsg")
db_errmsg :: proc(db: Db) -> cstring ---
@(link_name = "sqlite3_exec")
db_exec :: proc(db: Db, sql: cstring, callback: rawptr, callback_arg: rawptr, errmsg: ^cstring) -> c.int ---
@(link_name = "sqlite3_prepare_v2")
prepare_v2 :: proc(db: Db, sql: cstring, nByte: c.int, ppStmt: ^Stmt, pzTail: ^cstring) -> c.int ---
@(link_name = "sqlite3_step")
step :: proc(stmt: Stmt) -> c.int ---
@(link_name = "sqlite3_finalize")
finalize :: proc(stmt: Stmt) -> c.int ---
@(link_name = "sqlite3_column_text")
column_text :: proc(stmt: Stmt, iCol: c.int) -> cstring ---
@(link_name = "sqlite3_column_bytes")
column_bytes :: proc(stmt: Stmt, iCol: c.int) -> c.int ---
@(link_name = "sqlite3_bind_text")
bind_text :: proc(stmt: Stmt, idx: c.int, val: cstring, n: c.int, destructor: rawptr) -> c.int ---
@(link_name = "sqlite3_changes")
changes :: proc(db: Db) -> c.int ---
@(link_name = "sqlite3_serialize")
serialize :: proc(db: Db, zSchema: cstring, piSize: ^i64, mFlags: u32) -> [^]u8 ---
@(link_name = "sqlite3_deserialize")
deserialize :: proc(db: Db, zSchema: cstring, pData: [^]u8, szDb: i64, szBuf: i64, mFlags: DESERIALIZE_FLAGS) -> c.int ---
@(link_name = "sqlite3_malloc64")
@(link_name="sqlite3_open")
db_open :: proc(filename: cstring, ppDb: ^^rawptr) -> c.int ---
@(link_name="sqlite3_close")
db_close :: proc(db: ^rawptr) -> c.int ---
@(link_name="sqlite3_errmsg")
db_errmsg :: proc(db: ^rawptr) -> cstring ---
@(link_name="sqlite3_exec")
db_exec :: proc(db: ^rawptr, sql: cstring, callback: rawptr, callback_arg: rawptr, errmsg: ^cstring) -> c.int ---
@(link_name="sqlite3_prepare_v2")
prepare_v2 :: proc(db: ^rawptr, sql: cstring, nByte: c.int, ppStmt: ^^rawptr, pzTail: ^cstring) -> c.int ---
@(link_name="sqlite3_step")
step :: proc(stmt: ^rawptr) -> c.int ---
@(link_name="sqlite3_finalize")
finalize :: proc(stmt: ^rawptr) -> c.int ---
@(link_name="sqlite3_column_text")
column_text :: proc(stmt: ^rawptr, iCol: c.int) -> cstring ---
@(link_name="sqlite3_column_bytes")
column_bytes :: proc(stmt: ^rawptr, iCol: c.int) -> c.int ---
@(link_name="sqlite3_bind_text")
bind_text :: proc(stmt: ^rawptr, idx: c.int, val: cstring, n: c.int, destructor: rawptr) -> c.int ---
@(link_name="sqlite3_changes")
changes :: proc(db: ^rawptr) -> c.int ---
@(link_name="sqlite3_serialize")
serialize :: proc(db: ^rawptr, zSchema: cstring, piSize: ^i64, mFlags: u32) -> [^]u8 ---
@(link_name="sqlite3_deserialize")
deserialize :: proc(db: ^rawptr, zSchema: cstring, pData: [^]u8, szDb: i64, szBuf: i64, mFlags: u32) -> c.int ---
@(link_name="sqlite3_malloc64")
malloc64 :: proc(n: i64) -> [^]u8 ---
@(link_name = "sqlite3_free")
@(link_name="sqlite3_free")
free :: proc(p: rawptr) ---
}

View File

@@ -12,6 +12,24 @@ Ed25519Keypair :: struct {
Private: [32]u8,
}
read_wire_string :: proc(data: []u8, offset: ^int) -> (s: string, ok: bool) {
if offset^ + 4 > len(data) {
return
}
length := u32(data[offset^]) << 24 | u32(data[offset^ + 1]) << 16 |
u32(data[offset^ + 2]) << 8 | u32(data[offset^ + 3])
offset^ += 4
if offset^ + int(length) > len(data) {
return
}
s = string(data[offset^ : offset^ + int(length)])
offset^ += int(length)
ok = true
return
}
parse_ssh_public_key :: proc(pub_path: string) -> (pub: [32]u8, ok: bool) {
data, err := os.read_entire_file_from_path(pub_path, context.temp_allocator)
if err != nil {
@@ -235,21 +253,3 @@ is_encrypted_key :: proc(priv_path: string) -> bool {
return ciphername != "none"
}
read_wire_string :: proc(data: []u8, offset: ^int) -> (s: string, ok: bool) {
if offset^ + 4 > len(data) {
return
}
length := u32(data[offset^]) << 24 | u32(data[offset^ + 1]) << 16 |
u32(data[offset^ + 2]) << 8 | u32(data[offset^ + 3])
offset^ += 4
if offset^ + int(length) > len(data) {
return
}
s = string(data[offset^ : offset^ + int(length)])
offset^ += int(length)
ok = true
return
}

View File

@@ -1,11 +1,9 @@
#+test
package main
import "core:fmt"
import "core:os"
import "core:testing"
TEST_KEY_DIR :: "fixtures" + os.Path_Separator_String + "keys"
TEST_KEY_DIR :: "fixtures/keys"
@(test)
test_parse_ed25519_public_key :: proc(t: ^testing.T) {

View File

@@ -1,36 +1,89 @@
package main
import "core:text/table"
import "core:unicode/utf8"
import "core:encoding/json"
import "core:fmt"
import "core:io"
import "core:strings"
decorations := table.Decorations {
"┌",
"┬",
"┐",
"├",
"┼",
"┤",
"└",
"┴",
"┘",
"│",
"─",
}
render_table :: proc(w: io.Writer, headers: []string, rows: [][]string) {
col_widths := make([dynamic]int, 0, len(headers))
for i in 0 ..< len(headers) {
append(&col_widths, strings.rune_count(headers[i]))
}
for r in rows {
for i in 0 ..< len(r) {
rw := strings.rune_count(r[i])
if i < len(col_widths) && rw > col_widths[i] {
col_widths[i] = rw
}
}
}
// TODO: Optimize ansi_aware_width
ansi_aware_width :: proc(str: string) -> int {
buf: [4096]byte
pos := 0
i := 0
for i < len(str) {
if i + 1 < len(str) && str[i] == 0x1b && str[i + 1] == '[' {
i += 2
for i < len(str) {c := str[i]; i += 1; if c >= 0x40 && c <= 0x7E {break}}
b: strings.Builder
strings.builder_init(&b)
defer strings.builder_destroy(&b)
defer delete(col_widths)
hline :: proc(w: io.Writer, b: ^strings.Builder, left, mid, right: string, widths: [dynamic]int) {
strings.write_string(b, left)
for i in 0 ..< len(widths) {
for _ in 0 ..< widths[i] + 2 {
strings.write_string(b, "\u2500")
}
if i < len(widths) - 1 {
strings.write_string(b, mid)
} else {
buf[pos] = str[i]; pos += 1; i += 1
strings.write_string(b, right)
}
}
_, _, width := utf8.grapheme_count(string(buf[:pos]))
return width
fmt.wprintf(w, "%s\n", strings.to_string(b^), flush = false)
strings.builder_reset(b)
}
hline(w, &b, "\u250c", "\u252c", "\u2510", col_widths)
cell :: proc(b: ^strings.Builder, s: string, width: int) {
extra := len(s) - strings.rune_count(s)
fmt.sbprintf(b, " %-*s \u2502", width + extra, s)
}
strings.write_string(&b, "\u2502")
for i in 0 ..< len(headers) {
cell(&b, headers[i], col_widths[i])
}
fmt.wprintf(w, "%s\n", strings.to_string(b), flush = false)
strings.builder_reset(&b)
hline(w, &b, "\u251c", "\u253c", "\u2524", col_widths)
for r in rows {
strings.write_string(&b, "\u2502")
for i in 0 ..< len(r) {
cell(&b, r[i], col_widths[i])
}
fmt.wprintf(w, "%s\n", strings.to_string(b), flush = false)
strings.builder_reset(&b)
}
hline(w, &b, "\u2514", "\u2534", "\u2518", col_widths)
}
render_json_rows :: proc(w: io.Writer, headers: []string, rows: [][]string) {
entries := make([dynamic]map[string]string, 0, len(rows), context.temp_allocator)
for row in rows {
entry := make(map[string]string, len(headers), context.temp_allocator)
for i in 0 ..< len(headers) {
entry[headers[i]] = row[i]
}
append(&entries, entry)
}
data, err := json.marshal(entries[:], allocator = context.temp_allocator)
if err != nil {
fmt.eprintf("Error marshaling JSON: %v\n", err)
return
}
fmt.wprintf(w, "%s", data, flush = false)
}

View File

@@ -1,33 +1,198 @@
#+test
package main
import "core:encoding/json"
import "core:fmt"
import "core:strings"
import "core:testing"
@(test)
test_ansi_aware_width_plain_ascii :: proc(t: ^testing.T) {
testing.expect_value(t, ansi_aware_width("hello"), 5)
test_render_json_rows_normal :: proc(t: ^testing.T) {
b: strings.Builder
strings.builder_init(&b)
defer strings.builder_destroy(&b)
headers := []string{"name", "path"}
rows := [][]string{{"foo", "/home/user/.env"}, {"bar", "/home/user/project/.env"}}
w := strings.to_writer(&b)
render_json_rows(w, headers, rows)
output := strings.to_string(b)
result: []map[string]string = ---
unmarshal_err := json.unmarshal_string(output, &result, allocator = context.temp_allocator)
testing.expect(
t,
unmarshal_err == nil,
fmt.tprintf("json unmarshal failed: %v\noutput was: %q", unmarshal_err, output),
)
testing.expect(t, len(result) == 2, fmt.tprintf("expected 2 rows, got %d", len(result)))
testing.expect(
t,
result[0]["name"] == "foo",
fmt.tprintf("expected name=foo, got %q", result[0]["name"]),
)
testing.expect(t, result[0]["path"] == "/home/user/.env")
testing.expect(t, result[1]["name"] == "bar")
testing.expect(t, result[1]["path"] == "/home/user/project/.env")
}
@(test)
test_ansi_aware_width_empty :: proc(t: ^testing.T) {
testing.expect_value(t, ansi_aware_width(""), 0)
test_render_json_rows_special_chars :: proc(t: ^testing.T) {
b: strings.Builder
strings.builder_init(&b)
defer strings.builder_destroy(&b)
headers := []string{"key", "value"}
rows := [][]string {
{"quote", `has "double quotes"`},
{"backslash", `path\to\file`},
{"newline", "line1\nline2"},
{"mixed", `a "b" c\nd`},
}
w := strings.to_writer(&b)
render_json_rows(w, headers, rows)
output := strings.to_string(b)
result: []map[string]string = ---
unmarshal_err := json.unmarshal(
transmute([]byte)output,
&result,
allocator = context.temp_allocator,
)
testing.expect(
t,
unmarshal_err == nil,
fmt.tprintf("json unmarshal failed: %v\noutput was: %q", unmarshal_err, output),
)
testing.expect(t, len(result) == 4)
testing.expect(
t,
result[0]["value"] == `has "double quotes"`,
fmt.tprintf("got %q", result[0]["value"]),
)
testing.expect(t, result[1]["value"] == `path\to\file`)
testing.expect(t, result[2]["value"] == "line1\nline2")
testing.expect(t, result[3]["value"] == `a "b" c\nd`)
}
@(test)
test_ansi_aware_width_with_color_codes :: proc(t: ^testing.T) {
colored := COLOR_TABLE_HEADING + "Directory" + ANSI_RESET
testing.expect_value(t, ansi_aware_width(colored), 9)
test_render_json_rows_empty :: proc(t: ^testing.T) {
b: strings.Builder
strings.builder_init(&b)
defer strings.builder_destroy(&b)
headers := []string{"name"}
rows: [][]string
w := strings.to_writer(&b)
render_json_rows(w, headers, rows)
output := strings.to_string(b)
result: []map[string]string = ---
unmarshal_err := json.unmarshal_string(output, &result, allocator = context.temp_allocator)
testing.expect(
t,
unmarshal_err == nil,
fmt.tprintf("json unmarshal failed: %v\noutput was: %q", unmarshal_err, output),
)
testing.expect(t, len(result) == 0)
}
@(test)
test_ansi_aware_width_unicode :: proc(t: ^testing.T) {
testing.expect_value(t, ansi_aware_width("\u2713 Available"), 11)
testing.expect_value(t, ansi_aware_width("\u2717 Missing"), 9)
test_render_table_normal :: proc(t: ^testing.T) {
b: strings.Builder
strings.builder_init(&b)
defer strings.builder_destroy(&b)
headers := []string{"Name", "Path"}
rows := [][]string{{"foo", "/home/user/.env"}, {"bar", "/home/user/project/.env"}}
w := strings.to_writer(&b)
render_table(w, headers, rows)
output := strings.to_string(b)
expected := `┌──────┬─────────────────────────┐
│ Name │ Path │
├──────┼─────────────────────────┤
│ foo │ /home/user/.env │
│ bar │ /home/user/project/.env │
└──────┴─────────────────────────┘
`
testing.expect(
t,
output == expected,
fmt.tprintf(
"table output mismatch\n--- expected ---\n%s\n--- got ---\n%s\n",
expected,
output,
),
)
}
@(test)
test_ansi_aware_width_multiple_escape_sequences :: proc(t: ^testing.T) {
colored := COLOR_TABLE_HEADING + "a" + ANSI_RESET + "b" + COLOR_TABLE_HEADING + "c" + ANSI_RESET
testing.expect_value(t, ansi_aware_width(colored), 3)
test_render_table_empty :: proc(t: ^testing.T) {
b: strings.Builder
strings.builder_init(&b)
defer strings.builder_destroy(&b)
headers := []string{"Name"}
rows: [][]string
w := strings.to_writer(&b)
render_table(w, headers, rows)
output := strings.to_string(b)
expected := `┌──────┐
│ Name │
├──────┤
└──────┘
`
testing.expect(
t,
output == expected,
fmt.tprintf(
"table output mismatch\n--- expected ---\n%s\n--- got ---\n%s\n",
expected,
output,
),
)
}
@(test)
test_render_table_unicode :: proc(t: ^testing.T) {
b: strings.Builder
strings.builder_init(&b)
defer strings.builder_destroy(&b)
headers := []string{"Status", "Detail"}
rows := [][]string{{"\u2713 Available", "ok"}, {"\u2717 Missing", "fail"}}
w := strings.to_writer(&b)
render_table(w, headers, rows)
output := strings.to_string(b)
expected := `┌─────────────┬────────┐
│ Status │ Detail │
├─────────────┼────────┤
│ ✓ Available │ ok │
│ ✗ Missing │ fail │
└─────────────┴────────┘
`
testing.expect(
t,
output == expected,
fmt.tprintf(
"table output mismatch\n--- expected ---\n%s\n--- got ---\n%s\n",
expected,
output,
),
)
}

Binary file not shown.