on main help sign in
feat(shell): add jq as a bash builtin + a jq skill for it

Also, two local skills now exist for Crush development: * builtin-skills, for adding new builtin skills * shell-builtins, for adding new shell builtins

commit3773b23cb44afc4567d892f01c41c8dc62556ca0 authorChristian Rocha <christian@rocha.is> dateWed 15 Apr 2026, 21:16 (6 months ago) parentb37649e
8 files changed+503
.agents/skills/builtin-skills/SKILL.md+39 .agents/skills/shell-builtins/SKILL.md+51 go.mod+2 go.sum+4 internal/shell/jq.go+277 internal/shell/shell.go+20 internal/skills/builtin/jq/SKILL.md+97 internal/skills/skills_test.go+13
.agents/skills/builtin-skills/SKILL.mdnew file+39
@@ -0,0 +1,39 @@
1+---
2+name: builtin-skills
3+description: Guide for adding and modifying builtin skills embedded in the Crush binary. Use when creating a new builtin skill, editing an existing one, or understanding how the embedded skill system works (internal/skills/builtin/).
4+---
5+
6+# Builtin Skills
7+
8+Crush embeds skills directly into the binary via `internal/skills/builtin/`.
9+These are always available without user configuration.
10+
11+## How It Works
12+
13+- Each skill lives in `internal/skills/builtin/<skill-name>/SKILL.md`.
14+- The tree is embedded at compile time via `//go:embed builtin/*` in
15+ `internal/skills/embed.go`.
16+- `DiscoverBuiltin()` walks the embedded FS, parses each `SKILL.md`, and
17+ sets paths with the `crush://skills/` prefix (e.g.,
18+ `crush://skills/jq/SKILL.md`).
19+- The View tool resolves `crush://` paths from the embedded FS, not disk.
20+- User skills with the same name override builtins (last occurrence wins
21+ in `Deduplicate()`).
22+
23+## Adding a New Builtin Skill
24+
25+1. Create `internal/skills/builtin/<skill-name>/SKILL.md` with YAML
26+ frontmatter (`name`, `description`) and markdown instructions. The
27+ directory name must match the `name` field.
28+2. No extra wiring needed — `//go:embed builtin/*` picks up new
29+ directories automatically.
30+3. Add a test assertion in `TestDiscoverBuiltin` in
31+ `internal/skills/skills_test.go` to verify discovery.
32+4. Build and test: `go build . && go test ./internal/skills/...`
33+
34+## Existing Builtin Skills
35+
36+| Skill | Directory | Description |
37+|-------|-----------|-------------|
38+| `crush-config` | `builtin/crush-config/` | Crush configuration help |
39+| `jq` | `builtin/jq/` | jq JSON processor usage guide |
.agents/skills/shell-builtins/SKILL.mdnew file+51
@@ -0,0 +1,51 @@
1+---
2+name: shell-builtins
3+description: Guide for adding and modifying shell builtin commands in Crush's embedded shell (internal/shell/). Use when creating a new builtin command, editing an existing one, or understanding how commands are intercepted in the mvdan/sh interpreter.
4+---
5+
6+# Shell Builtins
7+
8+Crush's shell (`internal/shell/`) uses `mvdan.cc/sh/v3` for POSIX shell
9+emulation. Commands can be intercepted before they reach the OS by adding
10+**builtins** — functions handled in-process.
11+
12+## How Builtins Work
13+
14+Builtins live in `Shell.builtinHandler()` in `internal/shell/shell.go`.
15+This is an `interp.ExecHandlerFunc` middleware registered in
16+`execHandlers()` **before** the block handler, so builtins run even for
17+commands that would otherwise be blocked.
18+
19+The handler is a switch on `args[0]`. Each case either handles the command
20+inline or delegates to a helper function.
21+
22+## Adding a New Builtin
23+
24+1. **Add the case** to the switch in `builtinHandler()` in `shell.go`.
25+2. **Get I/O from the handler context**, not from `os.Stdin`/`os.Stdout`.
26+ This ensures the builtin works with pipes and redirections:
27+ ```go
28+ case "mycommand":
29+ hc := interp.HandlerCtx(ctx)
30+ return handleMyCommand(args, hc.Stdin, hc.Stdout, hc.Stderr)
31+ ```
32+3. **Implement the handler** in its own file (e.g.,
33+ `internal/shell/mycommand.go`). The function signature should accept
34+ args, stdin, stdout, and stderr:
35+ ```go
36+ func handleMyCommand(args []string, stdin io.Reader, stdout, stderr io.Writer) error {
37+ // args[0] is the command name ("mycommand"), args[1:] are arguments.
38+ // Write output to stdout, errors to stderr.
39+ // Return nil on success, or interp.ExitStatus(n) for non-zero exit codes.
40+ }
41+ ```
42+4. **Return values**: return `nil` for success, `interp.ExitStatus(n)` for
43+ non-zero exit codes. Write error messages to `stderr` before returning.
44+5. **No extra wiring needed** — `builtinHandler()` is already registered
45+ in `execHandlers()`.
46+
47+## Existing Builtins
48+
49+| Command | File | Description |
50+|---------|------|-------------|
51+| `jq` | `jq.go` | JSON processor using `github.com/itchyny/gojq` |
go.mod+2
@@ -44,6 +44,7 @@ require (
4444 github.com/go-git/go-git/v5 v5.17.2
4545 github.com/google/uuid v1.6.0
4646 github.com/invopop/jsonschema v0.13.0
47+ github.com/itchyny/gojq v0.12.19
4748 github.com/joho/godotenv v1.5.1
4849 github.com/jordanella/go-ansi-paintbrush v0.0.0-20240728195301-b7ad996ecf3d
4950 github.com/lucasb-eyer/go-colorful v1.4.0
@@ -142,6 +143,7 @@ require (
142143 github.com/gorilla/websocket v1.5.3 // indirect
143144 github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
144145 github.com/inconshreveable/mousetrap v1.1.0 // indirect
146+ github.com/itchyny/timefmt-go v0.1.8 // indirect
145147 github.com/jackmordaunt/icns/v3 v3.0.1 // indirect
146148 github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
147149 github.com/josharian/intern v1.0.0 // indirect
go.sum+4
@@ -250,6 +250,10 @@ github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2
250250github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
251251github.com/invopop/jsonschema v0.13.0 h1:KvpoAJWEjR3uD9Kbm2HWJmqsEaHt8lBUpd0qHcIi21E=
252252github.com/invopop/jsonschema v0.13.0/go.mod h1:ffZ5Km5SWWRAIN6wbDXItl95euhFz2uON45H2qjYt+0=
253+github.com/itchyny/gojq v0.12.19 h1:ttXA0XCLEMoaLOz5lSeFOZ6u6Q3QxmG46vfgI4O0DEs=
254+github.com/itchyny/gojq v0.12.19/go.mod h1:5galtVPDywX8SPSOrqjGxkBeDhSxEW1gSxoy7tn1iZY=
255+github.com/itchyny/timefmt-go v0.1.8 h1:1YEo1JvfXeAHKdjelbYr/uCuhkybaHCeTkH8Bo791OI=
256+github.com/itchyny/timefmt-go v0.1.8/go.mod h1:5E46Q+zj7vbTgWY8o5YkMeYb4I6GeWLFnetPy5oBrAI=
253257github.com/jackmordaunt/icns/v3 v3.0.1 h1:xxot6aNuGrU+lNgxz5I5H0qSeCjNKp8uTXB1j8D4S3o=
254258github.com/jackmordaunt/icns/v3 v3.0.1/go.mod h1:5sHL59nqTd2ynTnowxB/MDQFhKNqkK8X687uKNygaSQ=
255259github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 h1:BQSFePA1RWJOlocH6Fxy8MmwDt+yVQYULKfN0RoTN8A=
internal/shell/jq.gonew file+277
@@ -0,0 +1,277 @@
1+package shell
2+
3+import (
4+ "encoding/json"
5+ "fmt"
6+ "io"
7+ "os"
8+ "strings"
9+
10+ "github.com/itchyny/gojq"
11+ "mvdan.cc/sh/v3/interp"
12+)
13+
14+const jqUsage = `jq - Go implementation of jq (gojq 0.12.19 builtin)
15+
16+Synopsis:
17+ %% echo '{"foo": 128}' | jq '.foo'
18+
19+Usage:
20+ jq [OPTIONS] [FILTER] [FILE...]
21+
22+Options:
23+ -r, --raw-output output raw strings
24+ -j, --join-output implies -r with no newline delimiter
25+ -c, --compact-output output without pretty-printing
26+ -s, --slurp read all inputs into an array
27+ -n, --null-input use null as input value
28+ -e, --exit-status exit 1 when the last value is false or null
29+ -R, --raw-input read input as raw strings
30+ --arg name value set a string value to a variable
31+ --argjson name value set a JSON value to a variable
32+ -h, --help display this help
33+`
34+
35+// handleJQ implements the jq builtin using gojq. It supports a subset of jq
36+// flags: -r (raw output), -c (compact output), -s (slurp), -n (null input),
37+// -e (exit status), -R (raw input), and --arg name value.
38+//
39+// Note that this is somewhat of a reimplmentation of the CLI of the glorious
40+// github.com/itchyny/gojq, and we'd ideally get the CLI exposed upstream to
41+// avoid this falling out of sync.
42+func handleJQ(args []string, stdin io.Reader, stdout, stderr io.Writer) error {
43+ var (
44+ rawOutput bool
45+ compact bool
46+ slurp bool
47+ nullInput bool
48+ exitStatus bool
49+ rawInput bool
50+ joinOutput bool
51+ argNames []string
52+ argValues []any
53+ )
54+
55+ // Parse flags and extract the query.
56+ var queryStr string
57+ var fileArgs []string
58+ i := 1 // skip "jq"
59+ for i < len(args) {
60+ arg := args[i]
61+ switch {
62+ case arg == "-h" || arg == "--help":
63+ fmt.Fprint(stdout, jqUsage)
64+ return nil
65+ case arg == "-r" || arg == "--raw-output":
66+ rawOutput = true
67+ case arg == "-j" || arg == "--join-output":
68+ joinOutput = true
69+ rawOutput = true
70+ case arg == "-c" || arg == "--compact-output":
71+ compact = true
72+ case arg == "-s" || arg == "--slurp":
73+ slurp = true
74+ case arg == "-n" || arg == "--null-input":
75+ nullInput = true
76+ case arg == "-e" || arg == "--exit-status":
77+ exitStatus = true
78+ case arg == "-R" || arg == "--raw-input":
79+ rawInput = true
80+ case arg == "--arg":
81+ if i+2 >= len(args) {
82+ fmt.Fprintf(stderr, "jq: --arg requires name and value\n")
83+ return interp.ExitStatus(2)
84+ }
85+ argNames = append(argNames, "$"+args[i+1])
86+ argValues = append(argValues, args[i+2])
87+ i += 2
88+ case arg == "--argjson":
89+ if i+2 >= len(args) {
90+ fmt.Fprintf(stderr, "jq: --argjson requires name and value\n")
91+ return interp.ExitStatus(2)
92+ }
93+ var val any
94+ if err := json.Unmarshal([]byte(args[i+2]), &val); err != nil {
95+ fmt.Fprintf(stderr, "jq: invalid JSON for --argjson %s: %s\n", args[i+1], err)
96+ return interp.ExitStatus(2)
97+ }
98+ argNames = append(argNames, "$"+args[i+1])
99+ argValues = append(argValues, val)
100+ i += 2
101+ case arg == "--":
102+ i++
103+ // Remaining args are file arguments.
104+ for i < len(args) {
105+ fileArgs = append(fileArgs, args[i])
106+ i++
107+ }
108+ continue
109+ case strings.HasPrefix(arg, "-") && queryStr != "":
110+ fmt.Fprintf(stderr, "jq: unknown option: %s\n", arg)
111+ return interp.ExitStatus(2)
112+ default:
113+ if queryStr == "" {
114+ queryStr = arg
115+ } else {
116+ fileArgs = append(fileArgs, arg)
117+ }
118+ }
119+ i++
120+ }
121+
122+ if queryStr == "" {
123+ queryStr = "."
124+ }
125+
126+ query, err := gojq.Parse(queryStr)
127+ if err != nil {
128+ fmt.Fprintf(stderr, "jq: %s\n", err)
129+ return interp.ExitStatus(3)
130+ }
131+
132+ opts := []gojq.CompilerOption{
133+ gojq.WithEnvironLoader(os.Environ),
134+ }
135+ if len(argNames) > 0 {
136+ opts = append(opts, gojq.WithVariables(argNames))
137+ }
138+
139+ code, err := gojq.Compile(query, opts...)
140+ if err != nil {
141+ fmt.Fprintf(stderr, "jq: %s\n", err)
142+ return interp.ExitStatus(3)
143+ }
144+
145+ // Build input values.
146+ inputs, err := readInputs(stdin, fileArgs, nullInput, rawInput, slurp)
147+ if err != nil {
148+ fmt.Fprintf(stderr, "jq: %s\n", err)
149+ return interp.ExitStatus(2)
150+ }
151+
152+ var lastFalsy bool
153+ for _, input := range inputs {
154+ iter := code.Run(input, argValues...)
155+ for {
156+ v, ok := iter.Next()
157+ if !ok {
158+ break
159+ }
160+ if err, ok := v.(error); ok {
161+ fmt.Fprintf(stderr, "jq: %s\n", err)
162+ return interp.ExitStatus(5)
163+ }
164+ if exitStatus {
165+ lastFalsy = v == nil || v == false
166+ }
167+ if err := writeValue(stdout, v, rawOutput, compact, joinOutput); err != nil {
168+ return err
169+ }
170+ }
171+ }
172+
173+ if exitStatus && lastFalsy {
174+ return interp.ExitStatus(1)
175+ }
176+ return nil
177+}
178+
179+// readInputs reads JSON (or raw) input values from stdin or files.
180+func readInputs(stdin io.Reader, files []string, nullInput, rawInput, slurp bool) ([]any, error) {
181+ if nullInput {
182+ return []any{nil}, nil
183+ }
184+
185+ var readers []io.Reader
186+ if len(files) > 0 {
187+ for _, f := range files {
188+ file, err := os.Open(f)
189+ if err != nil {
190+ return nil, err
191+ }
192+ defer file.Close()
193+ readers = append(readers, file)
194+ }
195+ } else {
196+ readers = []io.Reader{stdin}
197+ }
198+
199+ var vals []any
200+ for _, r := range readers {
201+ data, err := io.ReadAll(r)
202+ if err != nil {
203+ return nil, err
204+ }
205+
206+ if rawInput {
207+ lines := strings.Split(string(data), "\n")
208+ if slurp {
209+ vals = append(vals, strings.Join(lines, "\n"))
210+ } else {
211+ for _, line := range lines {
212+ if line != "" || !slurp {
213+ vals = append(vals, line)
214+ }
215+ }
216+ }
217+ continue
218+ }
219+
220+ // Decode potentially multiple JSON values from the stream.
221+ dec := json.NewDecoder(strings.NewReader(string(data)))
222+ var streamVals []any
223+ for {
224+ var v any
225+ if err := dec.Decode(&v); err != nil {
226+ if err == io.EOF {
227+ break
228+ }
229+ return nil, fmt.Errorf("parse error: %w", err)
230+ }
231+ streamVals = append(streamVals, v)
232+ }
233+
234+ if slurp {
235+ vals = append(vals, streamVals)
236+ } else {
237+ vals = append(vals, streamVals...)
238+ }
239+ }
240+
241+ if len(vals) == 0 {
242+ return []any{nil}, nil
243+ }
244+ return vals, nil
245+}
246+
247+// writeValue writes a single jq output value.
248+func writeValue(w io.Writer, v any, raw, compact, join bool) error {
249+ if raw {
250+ if s, ok := v.(string); ok {
251+ if _, err := fmt.Fprint(w, s); err != nil {
252+ return err
253+ }
254+ if !join {
255+ _, err := fmt.Fprint(w, "\n")
256+ return err
257+ }
258+ return nil
259+ }
260+ }
261+
262+ var bs []byte
263+ var err error
264+ if compact {
265+ bs, err = gojq.Marshal(v)
266+ } else {
267+ bs, err = json.MarshalIndent(v, "", " ")
268+ }
269+ if err != nil {
270+ return err
271+ }
272+ if _, writeErr := w.Write(bs); writeErr != nil {
273+ return writeErr
274+ }
275+ _, err = fmt.Fprint(w, "\n")
276+ return err
277+}
internal/shell/shell.go+20
@@ -244,6 +244,25 @@ func (s *Shell) blockHandler() func(next interp.ExecHandlerFunc) interp.ExecHand
244244 }
245245}
246246
247+func (s *Shell) builtinHandler() func(next interp.ExecHandlerFunc) interp.ExecHandlerFunc {
248+ return func(next interp.ExecHandlerFunc) interp.ExecHandlerFunc {
249+ return func(ctx context.Context, args []string) error {
250+ if len(args) == 0 {
251+ return next(ctx, args)
252+ }
253+
254+ // Builtins.
255+ switch args[0] {
256+ case "jq":
257+ hc := interp.HandlerCtx(ctx)
258+ return handleJQ(args, hc.Stdin, hc.Stdout, hc.Stderr)
259+ default:
260+ return next(ctx, args)
261+ }
262+ }
263+ }
264+}
265+
247266// newInterp creates a new interpreter with the current shell state
248267func (s *Shell) newInterp(stdout, stderr io.Writer) (*interp.Runner, error) {
249268 return interp.New(
@@ -307,6 +326,7 @@ func (s *Shell) execStream(ctx context.Context, command string, stdout, stderr i
307326
308327func (s *Shell) execHandlers() []func(next interp.ExecHandlerFunc) interp.ExecHandlerFunc {
309328 handlers := []func(next interp.ExecHandlerFunc) interp.ExecHandlerFunc{
329+ s.builtinHandler(),
310330 s.blockHandler(),
311331 }
312332 if useGoCoreUtils {
internal/skills/builtin/jq/SKILL.mdnew file+97
@@ -0,0 +1,97 @@
1+---
2+name: jq
3+description: Process, filter, transform, generate, and construct JSON data using the jq command-line processor. Crush includes a built-in jq implementation (powered by gojq) available in the bash tool. Use when the user needs to query, filter, reshape, extract, create, or construct JSON — including API responses, config files, log output, or any structured data. Also use when generating JSON from scratch with jq -n, or when helping users write or debug jq expressions.
4+---
5+
6+# jq — Built-in JSON Processor
7+
8+Crush ships a built-in `jq` command (via `github.com/itchyny/gojq`) available
9+in the bash tool. No external binary is required.
10+
11+## Supported Flags
12+
13+| Flag | Description |
14+|------|-------------|
15+| `-r`, `--raw-output` | Output strings without quotes |
16+| `-j`, `--join-output` | Like `-r` but no trailing newline |
17+| `-c`, `--compact-output` | One-line JSON output |
18+| `-s`, `--slurp` | Read all inputs into an array |
19+| `-n`, `--null-input` | Use `null` as input (ignore stdin) |
20+| `-e`, `--exit-status` | Exit 1 if last output is `false` or `null` |
21+| `-R`, `--raw-input` | Read each line as a string, not JSON |
22+| `--arg name value` | Bind `$name` to a string value |
23+| `--argjson name value` | Bind `$name` to a parsed JSON value |
24+
25+File arguments after the filter are also supported: `jq '.foo' file.json`.
26+
27+## Differences from Standard jq
28+
29+The built-in uses gojq, which is a pure-Go jq implementation. Key
30+differences:
31+
32+- **No object key ordering** — keys are sorted by default; `keys_unsorted`
33+ and `-S` are unavailable.
34+- **Arbitrary-precision integers** — large integers keep full precision
35+ (addition, subtraction, multiplication, modulo, division when divisible).
36+- **String indexing** — `"abcde"[2]` returns `"c"`.
37+- **Not supported** — `--ascii-output`, `--seq`, `--stream`,
38+ `--stream-errors`, `-f`/`--from-file`, `--slurpfile`, `--rawfile`,
39+ `--args`, `--jsonargs`, `input_line_number`, `$__loc__`, some regex
40+ features (backreferences, look-around).
41+- **YAML** — gojq supports `--yaml-input`/`--yaml-output` but the
42+ built-in does not currently expose these flags.
43+
44+## Common Patterns
45+
46+Extract a field:
47+```sh
48+echo '{"name":"crush"}' | jq '.name'
49+```
50+
51+Filter an array:
52+```sh
53+echo '[1,2,3,4,5]' | jq '[.[] | select(. > 3)]'
54+```
55+
56+Reshape objects:
57+```sh
58+echo '{"first":"Ada","last":"Lovelace"}' | jq '{full: (.first + " " + .last)}'
59+```
60+
61+Use variables:
62+```sh
63+echo '{}' | jq --arg host localhost --argjson port 8080 '{host: $host, port: $port}'
64+```
65+
66+Slurp multiple JSON values:
67+```sh
68+echo '{"a":1}{"b":2}' | jq -s '.'
69+```
70+
71+Compact output for piping:
72+```sh
73+echo '{"a":1}' | jq -c '.a += 1'
74+```
75+
76+Raw string output:
77+```sh
78+echo '["one","two","three"]' | jq -r '.[]'
79+```
80+
81+Process a file:
82+```sh
83+jq '.dependencies | keys' package.json
84+```
85+
86+Null input for constructing JSON:
87+```sh
88+jq -n --arg msg hello '{"message": $msg}'
89+```
90+
91+## Tips
92+
93+- Pipe jq output to other commands: `jq -r '.url' data.json | xargs curl`
94+- Chain filters with `|` inside the expression, not shell pipes.
95+- Use `try` to suppress errors on missing keys: `jq 'try .foo.bar'`
96+- Use `// "default"` for fallback values: `jq '.name // "unknown"'`
97+- Use `@csv`, `@tsv`, `@base64`, `@html`, `@uri` for format strings.
internal/skills/skills_test.go+13
@@ -424,6 +424,19 @@ func TestDiscoverBuiltin(t *testing.T) {
424424 }
425425 }
426426 require.True(t, found, "crush-config builtin skill not found")
427+
428+ var foundJQ bool
429+ for _, s := range discovered {
430+ if s.Name == "jq" {
431+ foundJQ = true
432+ require.Equal(t, "crush://skills/jq/SKILL.md", s.SkillFilePath)
433+ require.Equal(t, "crush://skills/jq", s.Path)
434+ require.NotEmpty(t, s.Description)
435+ require.NotEmpty(t, s.Instructions)
436+ require.True(t, s.Builtin)
437+ }
438+ }
439+ require.True(t, foundJQ, "jq builtin skill not found")
427440}
428441
429442func TestDeduplicate(t *testing.T) {