1
1
mirror of https://github.com/wader/fq.git synced 2024-11-25 05:55:43 +03:00
Commit Graph

552 Commits

Author SHA1 Message Date
Mattias Wadman
77da7647ac interp: debug/* should use DEBUG: not DEBUG 2024-09-26 12:00:35 +02:00
Mattias Wadman
f8bb502e2c interp: Output raw strings for stderr/0
Same as jq

Also some cleanup and simplifications
2024-09-24 12:25:46 +02:00
Mattias Wadman
7b6d8532fd go,lint: Update to go 1.22 and fix some lint warnings 2024-09-12 11:44:38 +02:00
bump
f9dff5c65d Update github-golangci-lint to 1.60.2 from 1.60.1
Release notes https://github.com/golangci/golangci-lint/releases/tag/v1.60.2

Skip some gosec rules as they give too many false positives
2024-08-21 19:00:27 +02:00
Mattias Wadman
fc74f6857c intepr: Revert binary value each
Is a bad idea, behaves confusingly with .. etc. Maybe can be
done in the future via some option to make all raw bits behave
as arrays are instroduced.
2024-08-21 16:33:20 +02:00
Mattias Wadman
2a9dc432c2
Merge pull request #995 from wader/binary-each
intepr: Support each for binary values
2024-08-21 16:23:47 +02:00
Mattias Wadman
3f3183a5c5 intepr: Support each for binary values
Makes it possible to output all bytes or bits as integers in a binary value.
Ex: [1,2,3] | [tobytes[]] -> [1,2,3]
2024-08-21 16:17:19 +02:00
Mattias Wadman
550bcc27ff cli: Add go version to version string 2024-08-20 13:31:51 +02:00
Mattias Wadman
7bf112917b interp: Add bits_format=byte_array support
Represent binary values into an array of bytes when turn values into JSON
2024-08-20 08:43:22 +02:00
Mattias Wadman
824e51ec58 gojq: Update fq fork
Rebase fq changes on latester gojq main.

Most notable change visiable to users is that gojq now allows queries
as object literal values. For example this now works: {a: 1+2+3}

From upstream:
083fb39 refactor code using built-in min and max functions
470db58 bump up to Go 1.23, drop support for Go 1.20
0607aa5 bump up version to 0.12.16
0709341 update CHANGELOG.md for v0.12.16
1324e6e update dependencies
01355e9 improve parser to allow binary operators as object values
a41a5f8 fix debug/1 function to be available only when debug/0 is defined
f694c1b fix a benchmark test BenchmarkCompile
f2559f6 remove private compare function
0cd3a66 improve compiler to abort with error if query is missing
422cc9d refactor stringify function declarations of query
1130c4e refactor program body, rename rules, remove empty actions in parser
8d7ccc9 add tests for immutability of arrays
375e90d remove useless receivers
1b5ce7f set correct offset for multibyte tokens
8874f53 fix tests of exp10 and atan2 failing on some platforms
2024-08-19 14:15:02 +02:00
bump
2c38cf1324 Update github-golangci-lint to 1.60.1 from 1.59.1
Release notes https://github.com/golangci/golangci-lint/releases/tag/v1.60.1

Require go 1.21 and fix various lint warnings and use new features
2024-08-14 20:05:40 +02:00
Mattias Wadman
ebf063d1c0 doc: Cleanup and improve texts a bit 2024-04-30 14:16:13 +02:00
Mattias Wadman
6059b9ee1b fq: Use trim from gojq 2024-04-19 18:09:00 +02:00
Mattias Wadman
ee2ee24dbb decode: Cleanup io panic a bit 2024-04-12 17:23:03 +02:00
Mattias Wadman
c8f9cdc9d7 interp: Add string_truncate option
Was hardcoded to 50, now confiugurable also used affected by dd, dv etc
2024-04-08 17:45:10 +02:00
Mattias Wadman
bf7fa07c41 fq: Use go 1.20 and cleanup
Also rename *ex packages to *x
2024-04-01 19:14:10 +02:00
Mattias Wadman
48351028c5
Merge pull request #909 from wader/lint-more-and-fixes
lint: More linters and some fixes
2024-04-01 18:32:41 +02:00
Mattias Wadman
b542ff1d6c lint: More linters and some fixes 2024-04-01 18:24:41 +02:00
Mattias Wadman
621d7f2c22 decode: Align some heavily used structs to save space
From betteralign:
Compound: 8 bytes saved: struct with 40 pointer bytes could be 32
Value: 40 bytes saved: struct with 128 pointer bytes could be 88
2024-04-01 18:12:31 +02:00
Mattias Wadman
f5fd587350 gojq: Update fq fork
Fixes issue using ascii_downcase and ascii_upcase with decode value.

From upstream https://github.com/itchyny/gojq/blob/main/CHANGELOG.md:
* implement `ltrim`, `rtrim`, and `trim` functions
* implement `gojq.ParseError` for getting the offset and token of query parsing error
* implement `gojq.HaltError` for detecting halt errors and stopping outer iteration
* fix object construction with duplicate keys (`{x:0,y:1} | {a:.x,a:.y}`)
* fix `halt` and `halt_error` functions to stop the command execution immediately
* fix variable scope of binding syntax (`"a" as $v | def f: $v; "b" as $v | f`)
* fix pre-defined variables to be available in initial modules (`$ARGS` in `~/.jq`)
* fix `ltrimstr` and `rtrimstr` functions to emit error on non-string input
* fix `nearbyint` and `rint` functions to round ties to even
* improve parser to allow `reduce`, `foreach`, `if`, `try`-`catch` syntax as object values
* remove `pow10` in favor of `exp10`, define `scalbn` and `scalbln` by `ldexp`
2024-04-01 15:15:26 +02:00
Mattias Wadman
e26704049f gojq: Update fq fork
From upstream:
fix halt and halt_error to stop command immediately
2024-03-24 14:57:42 +01:00
Mattias Wadman
0db671f626 interp: Add todo test for eval error in path 2024-03-10 09:59:18 +01:00
Sam James
08df7f45c8
pkg/cli/test_exp.sh: use command -v
`which` is non-portable and not part of POSIX, but we can use `command -v`
for the same effect.

Debian [0] and Gentoo [1] are both trying to drop which from their base system.

[0] https://lwn.net/Articles/874049/
[1] https://bugs.gentoo.org/646588

Signed-off-by: Sam James <sam@gentoo.org>
2024-02-27 23:04:10 +00:00
Mattias Wadman
217660005e interp: Support ~/.config/fq as fallback on macOS
Got fed up with ~/Library/Application Support/fq
2024-02-09 16:10:38 +01:00
bump
888b0be069 Update github-golangci-lint to 1.56.0 from 1.55.2
Release notes https://github.com/golangci/golangci-lint/releases/tag/v1.56.0

Remove some nolint:errorlint not needed anymore
2024-02-07 17:31:49 +01:00
Mattias Wadman
69ae76593c interp: Fix crash when using line_bytes=0 2024-02-06 16:33:50 +01:00
Mattias Wadman
38b4412ad4 png: Type flags were off-by-one bit 2024-01-05 16:53:05 +01:00
Mattias Wadman
41226f48d8 readline: Switch to ergochat/readline
All fq changes merged.
2023-12-24 20:35:15 +01:00
Mattias Wadman
b05c7ec54f
Merge pull request #824 from mikez/leveldb
leveldb: Add LevelDB support
2023-12-09 16:47:43 +01:00
Mattias Wadman
7906f33d51 test: Support to more common -update flag 2023-12-06 11:47:38 +01:00
Michael B.
b05aa997ab leveldb: address PR comments 2023-12-05 11:31:30 +01:00
Mattias Wadman
dc376f3491 gojq: Update rebased fq fork
From upstream:
    implement pick function
    fix comment continuation with newline in dos format
    implement comment continuation with backslash
2023-11-05 18:04:09 +01:00
Mattias Wadman
29e75411ed interp: Fix infinite recursion when casting synthetic raw value into a jq value 2023-10-29 16:11:05 +01:00
Mattias Wadman
ac276ee119 gzip: Correctly handle multiple members
A gzip file consists of one or more "members" that are concatenated on uncompress.

Introduce a members array with each member and uncompressed is now the concatenation.

Fixes #794
2023-10-24 10:45:32 +02:00
Mattias Wadman
051a70bd4b interp: Change bit ranges to use exclusive end
All other ranges and slicing uses exclusive end so i think it make sense
to make it consistent.

Update docs and add additional example for non-byte-aligned field.

Also fixes issue showing zero bit ranges as start-NA.
2023-10-20 15:37:26 +02:00
Mattias Wadman
1d14ea51cd matroska: Decode ebml date type
Also some refactor of shared date decode functions to allow unit
2023-10-17 11:10:48 +02:00
Mattias Wadman
c28163f8f4 interp: Improve colors when using light background
Related to #729
2023-10-10 11:08:38 +02:00
Mattias Wadman
23ae4d978b decode,interp: Make synthetic values more visible and not act as decode values 2023-10-07 17:40:16 +02:00
Mattias Wadman
a614c9dfee interp: split: Correctly split binary
The thing splitted on should not be included.
2023-09-24 19:52:22 +02:00
Mattias Wadman
0699c80bac mp3_frame_xing: Detect lame ext more similar to ffmpeg and mediainfo 2023-09-08 12:09:35 +02:00
Mattias Wadman
c0352f2fd1 decode,interp: Don't shadow _key and error on missing _key
For fromjson and other "value" decode values fq make then behave both like
a normal jq value and decode value. This is to make tobytes, format etc work.
Before all _* would be treated as special keys. Now they are first looked up in
the wrapped value and then as decode values.

Also now ._key that don't exist reutrn null instead of throw error.

$ fq -n '`{"_format": 123}` | fromjson | ._format'
Now:
123
Before:
"json"

$ fq -n '`{}` | fromjson | ._missing'
Now:
null
Before
error
2023-09-03 10:47:59 +02:00
Mattias Wadman
0d0147643a gomod: Update x/exp and gomarkdown
Add cmp package from go 1.21 to have cmp.Compare to make sort easier
2023-08-09 14:36:51 +02:00
Mattias Wadman
d5345f0b8a cli: Rename --null/nul-output to --raw-output0
Remove -0 short arg, now it means the expression "-0".
This is to be in sync with jq 1.7 https://github.com/jqlang/jq/pull/2684

Correct and clarify that NUL and new lines are outputted after and not between each output.

Still missing is to abort on output containing zero when using --raw-output0
2023-08-08 16:33:39 +02:00
Mattias Wadman
fa3dba104b gojq: Update fq fork
Changes from upstream:
8fcc90e implement abs function
cbbd4bb fix default module paths assuming gojq is located in a bin directory
45d4c5b change behavior of walk with multiple outputs
846cf99 fix stderr to output strings in raw format
c352d50 change zero division to produce an error when dividend is zero
d579009 fix empty string repeating with the maximum integer
d4de65c fix string multiplication by zero to emit empty string
04b48fd remove deprecated leaf_paths function
c882861 bump up version to 0.12.13
c310d5d update CHANGELOG.md for v0.12.13
8fb5cc3 update dependencies
fb00b66 improve error message of indices, setpath, delpaths functions
2023-08-08 12:22:29 +02:00
Mattias Wadman
44f0060205 dev,jq: Reformat jq code to look more the same 2023-07-27 13:21:04 +02:00
Mattias Wadman
97194ad8bc mp4: Nicer major brand and handle some qt brand short strings better 2023-07-26 12:15:51 +02:00
Mattias Wadman
41f40b7f94 interp: Add to binary fast path for arrays with only 0-255 numbers and strings 2023-06-14 19:09:15 +02:00
Mattias Wadman
38cb82929a gojq: Update rebased fq fork
From upstream:
e940c43 update GitHub action dependencies
2d97ddb update jq links to the new jqlang organization
2a6cb00 improve error message of various functions
5e26bef add a note on differences of supported functions missing in jq
96c4843 fix functions returning arrays not to emit nil slices
4a681c8 improve .dockerignore
850177b Merge pull request #213 from movableink/uri-decode
dbba5e7 add urid to decode URI values, mirroring uri
2023-05-29 17:47:01 +02:00
Mattias Wadman
b2c0e5fcd8 interp: Make binary also respect bits_format 2023-05-17 15:10:32 +02:00
Mattias Wadman
93423d0da6
Merge pull request #673 from wader/bit-format-hex
interp: Add hex bits format
2023-05-15 17:49:15 +02:00