wader/fq
1
1
Fork 0
mirror of https://github.com/wader/fq.git synced 2024-12-29 00:22:38 +03:00
Code Issues Projects Releases Wiki Activity
1,948 Commits 36 Branches 23 Tags 62 MiB
bb2659d442
Commit Graph

493 Commits

Author SHA1 Message Date
Mattias Wadman
d838d2f613 zip: Add format decoder 2021-11-19 00:22:26 +01:00
Mattias Wadman
ffb5adff1d registry: Move to pkg/registry, feels better 2021-11-18 01:22:21 +01:00
Mattias Wadman
7af191d974 decode: Move io helper into *D 2021-11-18 01:17:41 +01:00
Mattias Wadman
6fba1a8125 decode: Fix bitbuf root handling a bit 2021-11-18 01:17:15 +01:00
Mattias Wadman
c083a9e475 decode: Fix MapRawToScalar regression 2021-11-17 17:14:19 +01:00
Mattias Wadman
986d5ecc50 decode: Move registry package to decode/registry and add a format group type 2021-11-17 16:59:56 +01:00
Mattias Wadman
5d98a6949a decode: Refactor Error/Fatal into printf functions 2021-11-17 16:26:13 +01:00
Mattias Wadman
ede2e77975 decode: Nicer scalar template and add doc 2021-11-17 16:13:10 +01:00
Mattias Wadman
826c5094f2 decode: Add Generated header for scalar 2021-11-17 10:19:10 +01:00
Mattias Wadman
b66ed32614 decode,interp: Make fuzzing work again and cleanup fatal/error code 2021-11-16 17:11:26 +01:00
Mattias Wadman
f9f866000e interp,decode: Add force option to ignore asserts 2021-11-16 13:03:56 +01:00
Mattias Wadman
5cd5633374 interp: Move jq function impls closer to where they belong 2021-11-16 11:43:11 +01:00
Mattias Wadman
996be0fae3 interp: Move more options code into options.jq 2021-11-16 10:49:18 +01:00
Mattias Wadman
3fafee8559 interp: Clearer help for -d 2021-11-16 08:43:29 +01:00
Mattias Wadman
1b32b42f93 decode: Major decode API refactor 
Generate more code
More generic and comfortable API
Improve and Update format decoder to new API
Add some more format tests
 2021-11-15 21:12:07 +01:00
Mattias Wadman
fcbfc29154 make: Move testjq.sh into pkg/interp 2021-11-12 00:23:16 +01:00
Mattias Wadman
178032e0a8 interp: Fix $opts shadowing in decode 2021-11-12 00:22:57 +01:00
Mattias Wadman
cc5f405554 interp,format: Update tests after decode refactor/tosym/toactual 2021-11-05 17:29:23 +01:00
Mattias Wadman
6a15625587 interp,decode: Refactor out Scalar from Value and merge Array/Struct into Compound 
Also add tosym, toactual
 2021-11-05 17:29:22 +01:00
Mattias Wadman
6cacc9b398 interp: Move *CtxWriter to ioextra, some comments cleanup 2021-11-05 17:28:29 +01:00
Mattias Wadman
ff2077bc0f interp: Simplify Function, aliases done in jq now 2021-11-05 17:28:29 +01:00
Mattias Wadman
93322bc434 interp: Implement format/0 in jq 2021-11-03 10:58:33 +01:00
Mattias Wadman
571bf29ab2 Change project title 2021-11-01 17:47:36 +01:00
Mattias Wadman
58ba84ff93 decode,interp: Add RecoverableErrorer interface instead of enumerate 2021-11-01 17:41:39 +01:00
Mattias Wadman
96cc1283cd interp: Eval options in jq instead of calling jq from go 
Simpler and causes less weird performance issues
 2021-11-01 17:05:36 +01:00
Mattias Wadman
13fae09172 interp: Don't print context cancel 2021-11-01 13:01:05 +01:00
Mattias Wadman
80b5d66c7a interp: Document inputs and repl/cli details 2021-11-01 13:01:05 +01:00
Mattias Wadman
0660ff08c6 interp: Clear up confusing --rawfile (add a jq alias) 2021-11-01 13:01:05 +01:00
Mattias Wadman
472c1ef081 interp: Refactor out string_input to own function 2021-11-01 11:40:24 +01:00
Mattias Wadman
0d693aa59e interp: Add line between usage and args help 2021-10-30 18:35:21 +02:00
Mattias Wadman
67898cb44a interp: Cleanup, use BufferRange for _open, progress for all decode 2021-10-30 01:17:14 +02:00
Mattias Wadman
88eade9ab9 ogg: Add flac support 
Refactor flac_streaminfo and flac_metadata from flac_metadatablocks
Some deocde bitbuf api cleanup
 2021-10-29 18:30:16 +02:00
Mattias Wadman
97f7317928 interp: Rework repl prompt code and fix some whitespace issues again 2021-10-28 12:08:19 +02:00
Mattias Wadman
cfdd92215d interp: Fix prompt issue with format 2021-10-27 16:40:39 +02:00
Mattias Wadman
80a69977a1 interp: Fix file leak and always cache for now 2021-10-26 11:50:59 +02:00
Mattias Wadman
826c8bd555 interp: Make include abs path work again 2021-10-26 11:02:30 +02:00
Mattias Wadman
2e964fa44f interp: Disable progress after decode is done 
Otherwise reading might cause progress output
 2021-10-25 11:11:26 +02:00
Mattias Wadman
bc1b3bf819 todo: Add note about symbols and iprint improvments 2021-10-22 11:01:48 +02:00
Mattias Wadman
b024316289 interp: Remove accidental extra space after path in prompt 2021-10-22 00:17:46 +02:00
Mattias Wadman
95ec5e1077 Cleanup and rename s/BufferView/BufferRange 2021-10-21 00:48:30 +02:00
Mattias Wadman
eedfd16b40 interp: Replace find with overloaded match that support buffer 2021-10-20 21:16:13 +02:00
Mattias Wadman
4af5739712 interp: Rework string/buffer for decode values 2021-10-20 01:31:59 +02:00
Mattias Wadman
58bf069ad5 interp: Return []byte value as a buffer for now 2021-10-19 15:44:48 +02:00
Mattias Wadman
5052bae18e decode,interp: Refactor to allow decode/fillgap a range 2021-10-19 14:38:11 +02:00
Mattias Wadman
6ff5acae03 gif: Support GIF87a 2021-10-19 01:53:37 +02:00
Mattias Wadman
8eaba88a10 decode: Refactor walk code a bit, add WalkRoot* to stay inside one root 2021-10-18 15:06:08 +02:00
Mattias Wadman
16d1f45de1 interp: find buffer should always use ByteRuneReader 
Fixes grep and bgrep with buffer
 2021-10-18 01:34:05 +02:00
Mattias Wadman
d6d94842b5 interp: Remove --options, probably just confusing 2021-10-18 00:59:35 +02:00
Mattias Wadman
36fd74a181 Add comment how raw byte regexp matching works 2021-10-17 14:48:50 +02:00
Mattias Wadman
7f36f703dd Add to/bytes/bits[range] 2021-10-17 14:43:37 +02:00
Mattias Wadman
7298a4cd8d interp: Add buffer match support to find and grep 2021-10-17 01:26:30 +02:00
Mattias Wadman
e86b45bd1a Add *grep/1/2 and find/1/2 2021-10-13 01:27:04 +02:00
Mattias Wadman
f0233bb811 test: Refactor out diff testing into own package 2021-10-10 17:20:49 +02:00
Mattias Wadman
879bb569f3 interp: Correctly check if _decode_value, add more has/1 tests 2021-10-07 13:35:48 +02:00
Mattias Wadman
691688022f fqtest: Refactor our script part to own package 
Hopefully can be reused for docs generation
 2021-10-06 18:49:03 +02:00
Mattias Wadman
1c3c65b191 interp: Rework buffer, still confusing 2021-10-05 22:26:05 +02:00
Mattias Wadman
b641c77216 interp: Cleanup unfinished/broken preview 2021-10-02 20:42:11 +02:00
Mattias Wadman
ff143d0bdb interp: Remove redundant decode arg 2021-10-02 10:53:01 +02:00
Mattias Wadman
20021f48ec bitio: Handle < 0 nbits 
Read/Write64 panic on < 0 bits
Readers return error
 2021-09-29 11:53:21 +02:00
Mattias Wadman
aa38ccf0e1 format,interp: Use MustGroup and add probe order test 2021-09-28 13:08:22 +02:00
Mattias Wadman
13e98d44be lint: Fix unused bufferRange and toBufferView 2021-09-28 02:07:44 +02:00
Mattias Wadman
618c1ea79f interp: Refactor/Rename BufferView 
TODO: fix bit level dump again
 2021-09-28 01:46:32 +02:00
Mattias Wadman
c997536f14 interp: Add root, buffer_root, format_root, parent and parents 2021-09-28 01:46:29 +02:00
Mattias Wadman
f1fcbe5dfb interp: Make has/1 work for _ext keys 2021-09-28 01:45:27 +02:00
Mattias Wadman
567bc4b342 interp: Rename to* funcs to match 2021-09-27 23:03:53 +02:00
Mattias Wadman
ab8080f495 make: Reorganize test targets 
Fix whitespaces in tests
 2021-09-27 22:47:17 +02:00
Mattias Wadman
3ff0c9b5e0 lint: Enable errcheck adnd revive 2021-09-27 11:01:14 +02:00
Mattias Wadman
00b7c18b65 doc: Some more doc work 2021-09-24 15:41:23 +02:00
Mattias Wadman
cf26b1fc8e interp: _readline: Use _repeat_break, add test 2021-09-23 14:39:03 +02:00
Mattias Wadman
0cce5ec61f interp: Reorganize, move out repl and options, more functions to funcs.jq 2021-09-22 21:08:36 +02:00
Mattias Wadman
110c86b959 interp: Fix broken dynamic width/height 
Refactor options to fixed/dynamic and just use stack
Added support for env in fqtest readlines
 2021-09-22 18:18:57 +02:00
Mattias Wadman
528e6b91ab interp: Refactor and use mapstructure 2021-09-22 01:35:27 +02:00
Mattias Wadman
6ce4ba919b interp: Use snake_case for all options 2021-09-22 00:32:57 +02:00
Mattias Wadman
bf19588e35 interp: Add ansi helper 2021-09-21 17:34:02 +02:00
Mattias Wadman
3e7e133047 interp: Move formats func def to jq 2021-09-21 16:42:35 +02:00
Mattias Wadman
ed21f36b23 mp3_frame: Rename samples_per_frame to sample_count 2021-09-20 18:54:20 +02:00
Mattias Wadman
976e992c31 interp: Move progress logic to jq 2021-09-20 17:47:40 +02:00
Mattias Wadman
d4142b875c decode,png: Add FieldFormatReaderLen, refactor out zlib to format 2021-09-19 20:49:15 +02:00
Mattias Wadman
3bfc95b3c2 test: Nicer test output 2021-09-19 11:58:37 +02:00
Mattias Wadman
90c19c6cc1 make,test: Move testjq to own script and reuse fq binary 
Remove version from --help for now
 2021-09-19 11:50:41 +02:00
Mattias Wadman
31d7611f55 fqtest: Add env support and isterminal/width/height support 2021-09-19 02:17:31 +02:00
Mattias Wadman
93fd097b53 cli: Add -M -C support and default to color if tty 2021-09-19 01:19:48 +02:00
Mattias Wadman
e4e269b162 Rename and move cli test to pkg/cli 2021-09-18 18:23:28 +02:00
Mattias Wadman
1fe5d95307 interp: Some better naming and typos 2021-09-18 10:47:43 +02:00
Mattias Wadman
3db11d3fa8 decode: Add UTF8Fn functions and trim some null terminated strings 2021-09-17 22:13:35 +02:00
Mattias Wadman
797bd4d652 flac: Refactor flac_metadatablock into flac_metadatablocks 
metadatablocks are not really used alone
 2021-09-17 15:46:13 +02:00
Mattias Wadman
ee972f490d dump: Add ascii header 2021-09-16 22:40:03 +02:00
Mattias Wadman
6ee7977c53 decode,format: Allow root array 2021-09-16 16:27:47 +02:00
Mattias Wadman
473b2243c9 decode: Simplify and move format arg into DecodeOptions 2021-09-16 15:29:11 +02:00
Mattias Wadman
26d615b852 decode: Move name/description into DecodeOptions 2021-09-16 14:42:32 +02:00
Mattias Wadman
c7416e6dcb decode, interp: More buffer reuse 2021-09-16 12:24:53 +02:00
Mattias Wadman
6207fcc8f0 decode: Pass context to be able to cancel properly 2021-09-16 10:32:29 +02:00
Mattias Wadman
8277b79c21 gojq: Update fq fork, support JQValue alt // 2021-09-16 00:46:25 +02:00
Mattias Wadman
c17483dbe0 decode: Fix accidental rename 2021-09-15 00:52:57 +02:00
Mattias Wadman
776a6b3136 decode: Reuse read buffer per decode to speed things up 2021-09-14 18:54:59 +02:00
Mattias Wadman
c770a754dc funcs: make in_bytes/bits_range more generic 2021-09-14 12:55:53 +02:00
Mattias Wadman
26c594feb1 input: make -R mimic jq on io error 2021-09-13 12:12:34 +02:00
Mattias Wadman
dcceaa4b35 repl: Fix help and error message a bit 2021-09-12 13:08:56 +02:00
Mattias Wadman
7423f450ef interp: Cleanup output types 2021-09-12 13:08:56 +02:00
Mattias Wadman
3d90b6d4b3 Remove fixed comment 2021-09-12 13:08:56 +02:00
Mattias Wadman
c9777aaded repl: completion in jq 2021-09-12 13:08:56 +02:00
Mattias Wadman
44d8b66a8e repl: Give error if repl is used in non-repl mode 2021-09-12 13:08:56 +02:00
Mattias Wadman
ec98fd315a cli: Make --argjson and --decode-file error similar 2021-09-12 13:08:56 +02:00
Mattias Wadman
1d15c1d0f2 gojq: Update fq fork 2021-09-12 13:08:56 +02:00
Mattias Wadman
4344b621da mp3: Only look for supported mp3 sync headers 2021-09-12 13:08:55 +02:00
Mattias Wadman
da386ea282 mp3: Be more relaxed with zero padding, just warn 2021-09-12 13:08:55 +02:00
Mattias Wadman
d350971fa2 cli: Add --decode-file VAR PATH support 2021-09-12 13:08:55 +02:00
Mattias Wadman
f5ffd32096 cli: Implement --arg, --argjson and --rawfile 2021-09-12 13:08:55 +02:00
Mattias Wadman
962d84dc68 funcs: Make intdiv truncate to int 2021-09-12 13:08:55 +02:00
Mattias Wadman
48517c7c60 cli: Unbreak part of completion 2021-09-12 13:08:55 +02:00
Mattias Wadman
f8e5944f91 dump,json: Properly figure if compound or not 2021-09-12 13:08:55 +02:00
Mattias Wadman
0a97f86f32 doc: Cleanup and note about repl limit 2021-09-12 13:08:55 +02:00
Mattias Wadman
a49e924d5f decode: Use golang.org/x/text for text decoding 2021-09-12 13:08:55 +02:00
Mattias Wadman
85d17199de cli: jq compat, multiple -L 2021-09-12 13:08:55 +02:00
Mattias Wadman
c8f0264682 cli: more jq compat 2021-09-12 13:08:55 +02:00
Mattias Wadman
ccf6cab491 repl: use map in _query_slurp_wrap 2021-09-12 13:08:55 +02:00
Mattias Wadman
538f4ffe71 cli: add -nul-output compat 2021-09-12 13:08:55 +02:00
Mattias Wadman
8cb380e7d9 fq: Generate decode alises code 2021-09-12 13:08:55 +02:00
Mattias Wadman
834f4a5893 fq: use jq functions for all display alises 2021-09-12 13:08:55 +02:00
Mattias Wadman
905c0ab74e fq: Add chunk/1 2021-09-12 13:08:55 +02:00
Mattias Wadman
25f5ad7566 fq,cli: Rename chunk to streaks, cleanup 2021-09-12 13:08:55 +02:00
Mattias Wadman
285356d21e fqtest: Cleanup and dont assert when WRITE_ACTUAL 2021-09-12 13:08:55 +02:00
Mattias Wadman
1d0ebb55e4 repl: Handle directives, add tests 2021-09-12 13:08:55 +02:00
Mattias Wadman
49f541c317 cli: Add proper repl iterator support 2021-09-12 13:08:54 +02:00
Mattias Wadman
c14c29a4a9 cli: Cleanup and more commens 2021-09-12 13:08:54 +02:00
Mattias Wadman
184df0a11b cli: A bit clearner array and iter prompt 2021-09-12 13:08:54 +02:00
Mattias Wadman
c7d45ff1e0 mp4: add pssh_playready format 2021-09-12 13:08:54 +02:00
Mattias Wadman
10d7ed7806 cli: Fix error filename on script error 2021-09-12 13:08:54 +02:00
Mattias Wadman
ac8cfca500 cli: Make --raw-string work with input/0 and inputs/0 
Also lots of refactor
 2021-09-12 13:08:54 +02:00
Mattias Wadman
2010caccd5 cli: Add error test 2021-09-12 13:08:54 +02:00
Mattias Wadman
569b631904 cli: Add output join tests 2021-09-12 13:08:54 +02:00
Mattias Wadman
3304f2956a cli: Add string_input options test 2021-09-12 13:08:54 +02:00
Mattias Wadman
17104f02d3 cli: Add --options to make help a bit nicer 2021-09-12 13:08:54 +02:00
Mattias Wadman
6356a84f15 cli: Add --raw-string 2021-09-12 13:08:54 +02:00
Mattias Wadman
4242bf6013 *_annexb: Refactor into avc/hevc_annexb 2021-09-12 13:08:54 +02:00
Mattias Wadman
9f55b6e619 decode: Refactor and add symbol struct and mapping 2021-09-12 13:08:53 +02:00
Mattias Wadman
f4b11b4be2 mpeg_annexb: Add format 2021-09-12 13:08:53 +02:00
Mattias Wadman
b33f2cd678 cli: Make profile build optional and move it to cli 2021-09-12 13:08:53 +02:00
Mattias Wadman
b849895970 fq: Add truncate array support to dump/display 2021-09-12 13:08:53 +02:00
Mattias Wadman
ba273be71e fq: Make relative include work with @builtin etc 2021-09-12 13:08:53 +02:00
Mattias Wadman
f828ae1319 gojq: Update gojq fq fork 
Removed some unneeded error changes in fork
 2021-09-12 13:08:53 +02:00
Mattias Wadman
dfcefc1b55 fq: Make format/0 native for performance 2021-09-12 13:08:53 +02:00
Mattias Wadman
45afbe6201 wip 2021-09-12 13:08:53 +02:00
Mattias Wadman
08ec4f0eeb funcs: Remove unsued string function 2021-09-12 13:08:53 +02:00
Mattias Wadman
dee10dbb78 fqtest: No need to escape empty stdout 2021-09-12 13:08:53 +02:00
Mattias Wadman
e2ff2a277a cli: Move help/0 to inter.jq, better help for -n 2021-09-12 13:08:53 +02:00
Mattias Wadman
d23edaa4a5 fq: Rename bits/0 bytes/0 to tobits tobytes, remove string/0 2021-09-12 13:08:53 +02:00
Mattias Wadman
518b7250e5 cli: Use format/0 to check if value is a format 2021-09-12 13:08:53 +02:00
Mattias Wadman
01d8a902eb funcs: Add delta/0 delta_by/0 2021-09-12 13:08:53 +02:00
Mattias Wadman
96f7a750b9 funcs: Fix typo add count/0 2021-09-12 13:08:53 +02:00
Mattias Wadman
2874bc7995 cli: Unbreak colors in windows 2021-09-12 13:08:53 +02:00
Mattias Wadman
75cf46f8f4 cli: Refactor options code 
Make nearly all options be just options and cli arguments
just sets various options.
 2021-09-12 13:08:53 +02:00
Mattias Wadman
5c8fb5f97a cli: Revert accidental history path change 2021-09-12 13:08:52 +02:00
Mattias Wadman
22eb53d5f6 cli: Cleanup MaybeLogFile 2021-09-12 13:08:52 +02:00
Mattias Wadman
6a8ba315fa number_to_bytes: Force int to make it work with bigint 2021-09-12 13:08:52 +02:00
Mattias Wadman
1888bb2892 gojq: Remove div operator 
Add intdiv function intead
 2021-09-12 13:08:52 +02:00
Mattias Wadman
e374b427f2 test: Less verbose 2021-09-12 13:08:52 +02:00
Mattias Wadman
b9b0326009 doc: fq - jq for files 2021-09-12 13:08:52 +02:00
Mattias Wadman
f600f2e54f build: Require go 1.17 2021-09-12 13:08:52 +02:00
Mattias Wadman
d9b45ba1ec internal: _global_var returns new value instead of _global_state 2021-09-12 13:08:52 +02:00
Mattias Wadman
80eaa4637e funcs: format helper 2021-09-12 13:08:52 +02:00
Mattias Wadman
d5c084c54a funcs: chunk_by comment 2021-09-12 13:08:52 +02:00
Mattias Wadman
54e121c72b interp: Refactor repl inputs a bit 2021-09-12 13:08:51 +02:00
Mattias Wadman
86b34a3f41 fqtest: Fix section regexp 2021-09-12 13:08:51 +02:00
Mattias Wadman
2684ed2561 cli: Prepare completion for better variables support 2021-09-12 13:08:51 +02:00
Mattias Wadman
e666380c4f cli: REPL and multiple files 2021-09-12 13:08:51 +02:00
Mattias Wadman
0dd848de6c cli: Better filenames in errors 2021-09-12 13:08:51 +02:00
Mattias Wadman
394e2b3837 cli: Only compelete at end or whitespace 2021-09-12 13:08:51 +02:00
Mattias Wadman
77f97aade2 builtin: Add chunk_by, count_by and debug 2021-09-12 13:08:51 +02:00
Mattias Wadman
f1507f7f65 mod: Use proper path and dont use replace 2021-09-12 13:08:50 +02:00
Mattias Wadman
1436fdc8ba completion: Better and _internal handling 2021-09-12 13:08:50 +02:00
Mattias Wadman
c155c89ba3 decode: Rename format *Decode to *Format 2021-09-12 13:08:50 +02:00
Mattias Wadman
2d4eb9ccc7 cli: Fix non-string variables and var(; f) variant to delete etc 2021-09-12 13:08:50 +02:00
Mattias Wadman
21bef18666 cli: User defined global vars 2021-09-12 13:08:50 +02:00
Mattias Wadman
0c7fa090b0 gojq: Initial update support 
decode value give no updateable error
JQValue json value tries to replicate behaveior
 2021-09-12 13:08:50 +02:00
Mattias Wadman
f8ab00eb84 cli: Remove unused eval debug arg 2021-09-12 13:08:50 +02:00
Mattias Wadman
f89329568a cli: rename fq.jq to interp.jq 2021-09-12 13:08:50 +02:00
Mattias Wadman
ae5566a6ce cli: Include paths and some refactor 2021-09-12 13:08:50 +02:00
Mattias Wadman
706b2f28e5 cli: Add exit 2 (like jq) for no args 2021-09-12 13:08:50 +02:00
Mattias Wadman
8d1fafe4b8 cli: Only show fq info for --help 2021-09-12 13:08:50 +02:00
Mattias Wadman
55cd45f675 cli: Simplify code 2021-09-12 13:08:49 +02:00
Mattias Wadman
8dc0f06b62 cli: Add completion tests 2021-09-12 13:08:49 +02:00
Mattias Wadman
2b8d11db20 cli: Fix indent 2021-09-12 13:08:49 +02:00
Mattias Wadman
78eb73724a cli: Nicer usage and indent input iteration 2021-09-12 13:08:49 +02:00
Mattias Wadman
3693667df4 Start of configurable json bit buffer formats 2021-09-12 13:08:49 +02:00
Mattias Wadman
f33b31049d Refactor decode.Value gojq bindings 2021-09-12 13:08:48 +02:00
Mattias Wadman
d0bb9a59f6 cli: Nicer grammar for --null-input help 2021-09-12 13:08:48 +02:00
Mattias Wadman
57f0ec1856 Improve cli help a bit 2021-09-12 13:08:48 +02:00
Mattias Wadman
d1be16784b Add decode struct each order test 2021-09-12 13:08:48 +02:00
Mattias Wadman
74bad2dca5 Fix broken value.fqtest 2021-09-12 13:08:48 +02:00
Mattias Wadman
1bd34bffc8 Same args error behavior as jq 2021-09-12 13:08:48 +02:00
Mattias Wadman
514739a149 Give proper error on missing short flag 2021-09-12 13:08:48 +02:00
Mattias Wadman
970465996c Init 2021-09-12 13:08:42 +02:00