Brian McGee
b86d0f897f
doc: configure hero and logo
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-05-02 11:41:36 +01:00
Brian McGee
397d051088
fix: nix filter for package
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-05-02 11:41:36 +01:00
Brian McGee
404b0a951a
Merge pull request 'improve caching and tracking' ( #34 ) from feat/improve-caching into main
...
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/34
2024-05-02 10:41:16 +00:00
Brian McGee
ed10f976f8
fix: fmt.Errorf formats
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-05-02 11:40:49 +01:00
Brian McGee
c720e414ac
chore: some cleanup and commenting
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-05-02 11:28:22 +01:00
Brian McGee
2eaf999a0e
feat: refactor some config init logic into config package
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-05-02 10:56:32 +01:00
Brian McGee
fb9493884c
chore: refactor logging initialisation
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-05-02 10:31:25 +01:00
Brian McGee
5a5c1ea03e
fix: record cache entries for files that don't match formatters
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-05-02 08:58:02 +01:00
Brian McGee
618f6f7e77
Merge pull request 'Stats output similar to treefmt.rs' ( #33 ) from feat/stats into main
...
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/33
Reviewed-by: Jonas Chevalier <zimbatm@noreply.git.numtide.com>
2024-05-01 12:49:32 +00:00
Brian McGee
04330a39fa
feat: add stats output similar to treefmt.rs
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-05-01 11:23:55 +01:00
Brian McGee
8e23c8602a
feat: update flake inputs
...
Moves us to go 1.22
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-05-01 11:23:55 +01:00
Brian McGee
d3cdb698e4
Merge pull request 'feat/configure-treefmt-pipeline' ( #32 ) from feat/configure-treefmt-pipeline into main
...
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/32
2024-05-01 10:20:10 +00:00
Brian McGee
757d5e746a
feat: configure a pipeline for formatting nix code
...
Relies on a fork of treefmt-nix until it can be merged.
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-05-01 09:12:01 +01:00
Brian McGee
3000c65ead
fix: nix-filter config for package
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-05-01 09:11:27 +01:00
Brian McGee
5d341f929f
Merge pull request 'feat: introduce concept of pipelines for better concurrency' ( #30 ) from feat/concurrency-refactor into main
...
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/30
2024-04-26 09:36:36 +00:00
Brian McGee
40b76b74a0
feat: ensure deterministic application of formatters
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-04-26 10:33:29 +01:00
Brian McGee
710efbd049
fix: remember timestamps from logging
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-04-26 09:55:09 +01:00
Brian McGee
fcce518d5e
feat: various perf improvements
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-04-25 12:16:04 +01:00
Brian McGee
6ae0e4f8e4
feat: add pipeline priority field
...
Allows for fine-grained control of execution order.
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-04-25 09:38:41 +01:00
Brian McGee
c71d69051a
feat: have each formatter filter paths again if part of a pipeline
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-04-25 09:17:51 +01:00
Brian McGee
8af5b3c076
feat: introduce concept of pipelines for better concurrency
...
Replaces the `Before` config option with an optional `Pipeline` key.
This is used to group formatters together in the order in which they are specified within the config file.
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-04-24 16:33:22 +01:00
Brian McGee
8333c99ebf
Merge pull request 'Implement Init and fix setting build variables' ( #26 ) from feat/init into main
...
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/26
Reviewed-by: Jonas Chevalier <zimbatm@noreply.git.numtide.com>
2024-02-28 09:19:37 +00:00
Brian McGee
49596b8e08
fix: setting build Name and Version
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-02-28 09:18:45 +00:00
Brian McGee
2ad87c2504
feat: implement init
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-02-28 09:18:45 +00:00
Jonas Chevalier
9de4fd4cf9
Merge pull request 'Feature parity with treefmt.rs' ( #22 ) from feat/explicit-paths-and-stdin into main
...
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/22
Reviewed-by: Jonas Chevalier <zimbatm@noreply.git.numtide.com>
2024-02-19 09:54:58 +00:00
Brian McGee
2b49923bf7
feat: add an example of 'before'
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-02-15 14:17:16 +00:00
Brian McGee
f4700c4e63
doc: update licence copyright year
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-02-15 14:00:47 +00:00
Brian McGee
cb8565d683
fix: reduce log verbosity
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-02-15 13:59:56 +00:00
Brian McGee
da82b80f29
feat: support --no-cache
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
diff --git a/cli/cli.go b/cli/cli.go
index 8b23262..b370ee7 100644
--- a/cli/cli.go
+++ b/cli/cli.go
@@ -11,6 +11,7 @@ var Cli = Format{}
type Format struct {
AllowMissingFormatter bool `default:"false" help:"Do not exit with error if a configured formatter is missing"`
WorkingDirectory kong.ChangeDirFlag `default:"." short:"C" help:"Run as if treefmt was started in the specified working directory instead of the current working directory"`
+ NoCache bool `help:"Ignore the evaluation cache entirely. Useful for CI"`
ClearCache bool `short:"c" help:"Reset the evaluation cache. Use in case the cache is not precise enough"`
ConfigFile string `type:"existingfile" default:"./treefmt.toml"`
FailOnChange bool `help:"Exit with error if any changes were made. Useful for CI."`
diff --git a/cli/format.go b/cli/format.go
index 6c46096..14ac16c 100644
--- a/cli/format.go
+++ b/cli/format.go
@@ -5,6 +5,7 @@ import (
"context"
"errors"
"fmt"
+ "io/fs"
"os"
"os/signal"
"strings"
@@ -168,6 +169,20 @@ func (f *Format) Run() error {
var changes int
+ processBatch := func() error {
+ if Cli.NoCache {
+ changes += len(batch)
+ } else {
+ count, err := cache.Update(batch)
+ if err != nil {
+ return err
+ }
+ changes += count
+ }
+ batch = batch[:0]
+ return nil
+ }
+
LOOP:
for {
select {
@@ -179,22 +194,17 @@ func (f *Format) Run() error {
}
batch = append(batch, path)
if len(batch) == batchSize {
- count, err := cache.Update(batch)
- if err != nil {
+ if err = processBatch(); err != nil {
return err
}
- changes += count
- batch = batch[:0]
}
}
}
// final flush
- count, err := cache.Update(batch)
- if err != nil {
+ if err = processBatch(); err != nil {
return err
}
- changes += count
if Cli.FailOnChange && changes != 0 {
return ErrFailOnChange
@@ -251,6 +261,22 @@ func (f *Format) Run() error {
}
defer close(pathsCh)
+
+ if Cli.NoCache {
+ return walker.Walk(ctx, func(path string, info fs.FileInfo, err error) error {
+ select {
+ case <-ctx.Done():
+ return ctx.Err()
+ default:
+ // ignore symlinks and directories
+ if !(info.IsDir() || info.Mode()&os.ModeSymlink == os.ModeSymlink) {
+ pathsCh <- path
+ }
+ return nil
+ }
+ })
+ }
+
return cache.ChangeSet(ctx, walker, pathsCh)
})
diff --git a/cli/format_test.go b/cli/format_test.go
index fb389fe..2349767 100644
--- a/cli/format_test.go
+++ b/cli/format_test.go
@@ -216,6 +216,15 @@ func TestCache(t *testing.T) {
as.NoError(err)
as.Contains(string(out), "0 files changed")
+ // clear cache
+ out, err = cmd(t, "--config-file", configPath, "--tree-root", tempDir, "-c")
+ as.NoError(err)
+ as.Contains(string(out), fmt.Sprintf("%d files changed", 29))
+
+ out, err = cmd(t, "--config-file", configPath, "--tree-root", tempDir)
+ as.NoError(err)
+ as.Contains(string(out), "0 files changed")
+
// no cache
out, err = cmd(t, "--config-file", configPath, "--tree-root", tempDir, "--no-cache")
as.NoError(err)
diff --git a/nix/packages.nix b/nix/packages.nix
index 127eb08..e0f8604 100644
--- a/nix/packages.nix
+++ b/nix/packages.nix
@@ -13,7 +13,7 @@
packages = rec {
treefmt = inputs'.gomod2nix.legacyPackages.buildGoApplication rec {
pname = "treefmt";
- version = "0.0.1+dev";
+ version = "2.0.0+dev";
# ensure we are using the same version of go to build with
inherit (pkgs) go;
diff --git a/cli/cli.go b/cli/cli.go
index 8b23262..b370ee7 100644
--- a/cli/cli.go
+++ b/cli/cli.go
@@ -11,6 +11,7 @@ var Cli = Format{}
type Format struct {
AllowMissingFormatter bool `default:"false" help:"Do not exit with error if a configured formatter is missing"`
WorkingDirectory kong.ChangeDirFlag `default:"." short:"C" help:"Run as if treefmt was started in the specified working directory instead of the current working directory"`
+ NoCache bool `help:"Ignore the evaluation cache entirely. Useful for CI"`
ClearCache bool `short:"c" help:"Reset the evaluation cache. Use in case the cache is not precise enough"`
ConfigFile string `type:"existingfile" default:"./treefmt.toml"`
FailOnChange bool `help:"Exit with error if any changes were made. Useful for CI."`
diff --git a/cli/format.go b/cli/format.go
index 6c46096..14ac16c 100644
--- a/cli/format.go
+++ b/cli/format.go
@@ -5,6 +5,7 @@ import (
"context"
"errors"
"fmt"
+ "io/fs"
"os"
"os/signal"
"strings"
@@ -168,6 +169,20 @@ func (f *Format) Run() error {
var changes int
+ processBatch := func() error {
+ if Cli.NoCache {
+ changes += len(batch)
+ } else {
+ count, err := cache.Update(batch)
+ if err != nil {
+ return err
+ }
+ changes += count
+ }
+ batch = batch[:0]
+ return nil
+ }
+
LOOP:
for {
select {
@@ -179,22 +194,17 @@ func (f *Format) Run() error {
}
batch = append(batch, path)
if len(batch) == batchSize {
- count, err := cache.Update(batch)
- if err != nil {
+ if err = processBatch(); err != nil {
return err
}
- changes += count
- batch = batch[:0]
}
}
}
// final flush
- count, err := cache.Update(batch)
- if err != nil {
+ if err = processBatch(); err != nil {
return err
}
- changes += count
if Cli.FailOnChange && changes != 0 {
return ErrFailOnChange
@@ -251,6 +261,22 @@ func (f *Format) Run() error {
}
defer close(pathsCh)
+
+ if Cli.NoCache {
+ return walker.Walk(ctx, func(path string, info fs.FileInfo, err error) error {
+ select {
+ case <-ctx.Done():
+ return ctx.Err()
+ default:
+ // ignore symlinks and directories
+ if !(info.IsDir() || info.Mode()&os.ModeSymlink == os.ModeSymlink) {
+ pathsCh <- path
+ }
+ return nil
+ }
+ })
+ }
+
return cache.ChangeSet(ctx, walker, pathsCh)
})
diff --git a/nix/packages.nix b/nix/packages.nix
index 127eb08..e0f8604 100644
--- a/nix/packages.nix
+++ b/nix/packages.nix
@@ -13,7 +13,7 @@
packages = rec {
treefmt = inputs'.gomod2nix.legacyPackages.buildGoApplication rec {
pname = "treefmt";
- version = "0.0.1+dev";
+ version = "2.0.0+dev";
# ensure we are using the same version of go to build with
inherit (pkgs) go;
2024-02-15 13:59:56 +00:00
Brian McGee
d53f98ea05
feat: support --version
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-02-15 13:59:56 +00:00
Brian McGee
d4ab015bc6
chore: remove internal directory
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-02-15 13:59:55 +00:00
Brian McGee
0fbae06f19
feat: support reading paths from stdin
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-02-15 10:37:56 +00:00
Brian McGee
4dd4c55ff5
feat: add nix direnv source to .envrc
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-02-14 15:15:39 +00:00
Brian McGee
6225883073
chore: update lock file
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-02-14 15:13:41 +00:00
Brian McGee
cb9564c7ef
feat: add -s -w build flags to reduce binary size
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-02-14 15:10:20 +00:00
Brian McGee
b109358490
feat: create config package ( #25 )
...
Move all config related code into a config package.
Signed-off-by: Brian McGee <brian@bmcgee.ie>
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/25
Reviewed-by: Jonas Chevalier <zimbatm@noreply.git.numtide.com>
Co-authored-by: Brian McGee <brian@bmcgee.ie>
Co-committed-by: Brian McGee <brian@bmcgee.ie>
2024-01-15 10:46:49 +00:00
Brian McGee
15db7f459c
fix: duplicate processing in ordered formatters ( #24 )
...
Fixes a bug with formatters processing paths out of order.
Signed-off-by: Brian McGee <brian@bmcgee.ie>
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/24
Co-authored-by: Brian McGee <brian@bmcgee.ie>
Co-committed-by: Brian McGee <brian@bmcgee.ie>
2024-01-12 15:15:51 +00:00
Brian McGee
c7d0138a02
support formatter ordering ( #20 )
...
Allows specifying the order in which formatters are applied.
Very simple for now, adding a `Before` field to the formatted config which allows the user to say that formatter `x` needs to be applied _before_ formatted `y`.
```toml
[formatter.statix]
command = "statix"
includes = ["*.nix"]
before = "deadnix"
[formatter.deadnix]
command = "statix"
includes = ["*.nix"]
```
Signed-off-by: Brian McGee <brian@bmcgee.ie>
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/20
Reviewed-by: Jonas Chevalier <zimbatm@noreply.git.numtide.com>
Co-authored-by: Brian McGee <brian@bmcgee.ie>
Co-committed-by: Brian McGee <brian@bmcgee.ie>
2024-01-12 11:46:04 +00:00
Brian McGee
80e99b6d75
feat: use go-git index instead of git ls-files
( #23 )
...
Figured out how to use `go-git` properly.
```console
# git
❯ nix run .# -- --config-file ./test/echo.toml --tree-root /home/brian/Development/com/github/nixos/nixpkgs -c
38539 files changed in 272.843495ms
# filesystem
❯ nix run .# -- --config-file ./test/echo.toml --tree-root /home/brian/Development/com/github/nixos/nixpkgs -c --walk filesystem
38567 files changed in 348.84277ms
```
Signed-off-by: Brian McGee <brian@bmcgee.ie>
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/23
Reviewed-by: Jonas Chevalier <zimbatm@noreply.git.numtide.com>
Co-authored-by: Brian McGee <brian@bmcgee.ie>
Co-committed-by: Brian McGee <brian@bmcgee.ie>
2024-01-12 11:33:14 +00:00
Brian McGee
5711caebb9
feat: support .gitignore files ( #19 )
...
Introduces a `--walk` flag which can be used to tell `treefmt` how to traverse the directory specified by `--tree-root`.
By default, it will attempt to use `git ls-files`. If this fails, it falls back to using the filesystem.
You can explicitly traverse the filesystem instead of using git by providing `--walk filesystem`.
Close #1
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/19
Reviewed-by: Jonas Chevalier <zimbatm@noreply.git.numtide.com>
Co-authored-by: Brian McGee <brian@bmcgee.ie>
Co-committed-by: Brian McGee <brian@bmcgee.ie>
2024-01-11 20:52:22 +00:00
Brian McGee
55ca4468cc
fix: stalling on large file sets ( #18 )
...
When running against nixpkgs we were stalling. This was due to a long running read tx which was preventing any writes.
This breaks up reading the cache when walking the filesystem into many smaller read txs.
On my laptop I'm now getting the following with the echo sample:
```console
# fresh cache
❯ nix run .# -- -c --config-file ./test/echo.toml --tree-root ../../../github.com/nixos/nixpkgs
38825 files changed in 320.655826ms
# hot cache
❯ nix run .# -- --config-file ./test/echo.toml --tree-root ../../../github.com/nixos/nixpkgs
0 files changed in 252.920853ms%
```
Signed-off-by: Brian McGee <brian@bmcgee.ie>
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/18
Reviewed-by: Jonas Chevalier <zimbatm@noreply.git.numtide.com>
Co-authored-by: Brian McGee <brian@bmcgee.ie>
Co-committed-by: Brian McGee <brian@bmcgee.ie>
2024-01-07 18:57:51 +00:00
Brian McGee
a3ca7825a6
feat: support changing work directory ( #15 )
...
Closes #10
Signed-off-by: Brian McGee <brian@bmcgee.ie>
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/15
Reviewed-by: Jonas Chevalier <zimbatm@noreply.git.numtide.com>
Co-authored-by: Brian McGee <brian@bmcgee.ie>
Co-committed-by: Brian McGee <brian@bmcgee.ie>
2024-01-03 15:21:45 +00:00
Brian McGee
84629f7b94
feat: support fail on change ( #16 )
...
Closes #8
Signed-off-by: Brian McGee <brian@bmcgee.ie>
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/16
Reviewed-by: Jonas Chevalier <zimbatm@noreply.git.numtide.com>
Co-authored-by: Brian McGee <brian@bmcgee.ie>
Co-committed-by: Brian McGee <brian@bmcgee.ie>
2024-01-03 13:10:54 +00:00
zimbatm
aebbcfd4f1
nit: clean batch capacity ( #17 )
...
Since we discussed this in https://git.numtide.com/numtide/treefmt/pulls/14#issuecomment-609
It doesn't really matter.
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/17
Co-authored-by: zimbatm <zimbatm@zimbatm.com>
Co-committed-by: zimbatm <zimbatm@zimbatm.com>
2024-01-03 10:39:55 +00:00
Brian McGee
4a78a1e319
fix: nix build
...
- update `gomod2nix.toml`
- add formatters as `nativeBuildInputs` for nix build
- export a `XDG_CACHE_HOME` for checks
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-01-03 09:20:32 +00:00
Brian McGee
811f883a2b
feat/bust-cache-validators-change ( #14 )
...
Tracks the mod time and size of a formatter's executable in bolt.
The cache is busted using the following criteria:
- a new formatter has been configured.
- an existing formatter has changed (mod time or size)
- an existing formatter has been removed from config
Also implemented better resolution of symlinks when determining a formatters executable path.
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/14
Reviewed-by: Jonas Chevalier <zimbatm@noreply.git.numtide.com>
Co-authored-by: Brian McGee <brian@bmcgee.ie>
Co-committed-by: Brian McGee <brian@bmcgee.ie>
2024-01-03 08:08:57 +00:00
Brian McGee
ada9a72a7e
feat: simplify some test code
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-01-02 14:57:21 +00:00
zimbatm
96b1560327
feat: support global excludes ( #13 )
...
Stacked on top of #11
Co-authored-by: Brian McGee <brian@bmcgee.ie>
Reviewed-on: https://git.numtide.com/numtide/treefmt/pulls/13
Co-authored-by: zimbatm <zimbatm@zimbatm.com>
Co-committed-by: zimbatm <zimbatm@zimbatm.com>
2024-01-02 12:12:47 +00:00
Brian McGee
d8d666a132
feat: use exec.LookPath to find formatter executable
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-01-02 10:36:51 +00:00
Brian McGee
26c2ae1cda
feat: hex encode db name
...
Signed-off-by: Brian McGee <brian@bmcgee.ie>
2024-01-02 10:36:48 +00:00