wader/fq
1
1
Fork 0
mirror of https://github.com/wader/fq.git synced 2024-11-23 18:56:52 +03:00
Code Issues Projects Releases Wiki Activity
1,404 Commits 33 Branches 23 Tags 59 MiB
56dcb3a0e3
Commit Graph

223 Commits

Author SHA1 Message Date
Mattias Wadman
7cea8757b1 doc: Add 0.0.6 changes 2022-03-11 17:01:26 +01:00
Mattias Wadman
dff7e7dadb interp: Cleanup binary regexp overloading and add explode 
Add explode for binary that explode into an array of the binary unit tobits
Remove scan_toend as it feels a bit weird and one can use tobytes | .[match(...):] instead
Some general cleanup also
 2022-03-09 20:17:12 +01:00
Mattias Wadman
b97776c9e3 doc: Add fq bts2022 presentation 2022-03-08 18:20:18 +01:00
Mattias Wadman
3f50bb90a6 interp: Rework formats and options help 
Replace --formats with --help formats
Add --help options
 2022-03-03 18:25:20 +01:00
Mattias Wadman
920629f5fd doc: Regenerate and fix macho section size 
Make macho format documentaion a bit more consistent.
 2022-03-03 09:55:53 +01:00
Mattias Wadman
0a043f9096 repl,interp: Refactor repl and slurp 
Now repl, slurp and help implemented using same query rewrite.
Include filename context in error if possible.
Add spew function that does opposite of slurp.
Start of help infra, not done or documented yet.
Show error pointer on parse error.
Rename internal eval to _eval and make eval be wrapper that
does rewrite and has various eror handling etc.
Nicer repl, slupr and help errors.
 2022-03-01 18:31:00 +01:00
Mattias Wadman
d3397cf9e7 doc: Tweak format diagram 2022-02-28 10:42:39 +01:00
Mattias Wadman
758b2d0e34 doc: Regenerate after macho merge 2022-02-28 10:28:21 +01:00
siddik.acil
2e7767cd05 macho: remake docs 2022-02-21 20:33:30 +01:00
Mattias Wadman
d334c2d4ea doc: Add href in supported format list 2022-02-19 01:33:45 +01:00
Mattias Wadman
c95b0d6d19 doc: Forgot make doc 2022-02-16 20:13:28 +01:00
Mattias Wadman
68ba2e810e doc: Add CHANGES.md documenting release changes 2022-02-13 23:36:11 +01:00
Mattias Wadman
d4cc544448
Merge pull request #153 from wader/doc-dev-more-decode-api 
doc: More decode API details and polish
 2022-02-13 17:22:04 +01:00
Mattias Wadman
b3504680f7 doc: More decode API details and polish 2022-02-13 16:28:59 +01:00
Mattias Wadman
dd3e40fb1c doc: Unbreak formats_digaram.jq since radix change 2022-02-13 10:31:53 +01:00
Xentripetal
d54011663f make doc 2022-02-12 22:17:32 -06:00
Xentripetal
9636613ec6 Merge branch 'master' of https://github.com/wader/fq into avro 2022-02-12 22:16:50 -06:00
Mattias Wadman
b86da7aecc doc: Add inital decoder API documentation 2022-02-12 16:13:29 +01:00
Mattias Wadman
48a19cb82c interp: Add paste function to allow pasting text into REPL etc 
Also refactor readline and eval args into option struct and partinally start
addressing some side effects during completion.
 2022-02-11 17:59:06 +01:00
Xentripetal
0300c955c5 Take heading off doc to match make doc format 2022-02-09 20:59:29 -06:00
Xentripetal
2605bce4e3 Lint and add basic doc 2022-02-09 20:46:12 -06:00
Mattias Wadman
36307857de doc: Cleanup and add more decode value and binary documentation 2022-02-09 21:08:01 +01:00
Mattias Wadman
06245d1295 binary,decode,doc: Rename buffer to binary and add some documentation 
Rename buffer to binary. Still some work left what to call buffer/binary in decode code.
Document decode value and binary type
Fix proper unit padding for tobytes and add still undocumenated extra padding argument.
Add some additional binary tests
 2022-02-08 22:20:28 +01:00
Xentripetal
5b11385ea4 Pull latest from wader 2022-02-07 17:35:33 -06:00
Mattias Wadman
0312c92c31 asn1_ber: Add more doc and multiple outputs for frompem 2022-02-07 17:41:05 +01:00
Mattias Wadman
0829c167cc asn1_ber: Add decoder 
Basic support, can also do CER and DER but without any extra validation.
No schema support.

Redo format doc.md usage a bit, now format/<dir>/<format>.md instead.

Related to #20
 2022-02-06 22:13:03 +01:00
Mattias Wadman
91b0fed929
Merge pull request #127 from wader/doc-foramts-option-links 
doc: Add format links to format table
 2022-02-05 18:24:03 +01:00
Mattias Wadman
c15f5283c0 doc: Add format links to format table 2022-02-05 18:15:18 +01:00
Mattias Wadman
366f6b185c interp: Support force decode as -o force=true 2022-02-05 17:24:42 +01:00
Mattias Wadman
7c5215347d bitio,decode: Refactor bitio usage and make buffer slicing more correct 
Remove bitio.Buffer layer. bitio.Buffer was a kitchen sink layer with helpers
now it's just a buffer and most functions have been moved to decode instead.

bitio package now only have primitive types and functions simialar to standard
library io and bytes packages.

Make nearly eveything internally use bitio.Bit* interfaces so that slicing work
correctly this will also make it possible to start experimenting with more
complicated silcing helpers, ex things like:
breplace(.header.bitrate; 123) to get a new buffer with bitrate changed.
 2022-02-04 21:41:53 +01:00
Mattias Wadman
710c29b24e doc: Color edges in diagram based on dest 2022-02-02 19:39:33 +01:00
Mattias Wadman
2ab395a03b protobuf: Add note about sub message decoding 
Fix and cleanup some other doc also
 2022-02-01 22:22:19 +01:00
Mattias Wadman
6b51b0676f doc: More display alias leftover fixes 2022-01-29 13:22:10 +01:00
Mattias Wadman
f0ce7179b2 doc: Document display and some more jq hints 
Also fixed some verbose -> dv leftovers
 2022-01-29 13:10:25 +01:00
Mattias Wadman
bc20f09fdb
Merge pull request #117 from wader/make-cleanup-targets 
make: Cleanup some not very used targets
 2022-01-29 12:31:10 +01:00
Mattias Wadman
b66e3287f2
Merge pull request #112 from wader/interp-cleanup-d-alises 
interp: Cleanup display aliases, now: d, da, dd, dv, ddv
 2022-01-29 12:28:46 +01:00
Mattias Wadman
fc52f20ed1
Merge pull request #116 from wader/doc-fix-filenames 
doc,matroska: Fix filesname in example
 2022-01-29 12:27:38 +01:00
Mattias Wadman
bf170be8ea make: Cleanup some not very used targets 2022-01-29 12:27:15 +01:00
Mattias Wadman
2efc3cba9b
Merge pull request #115 from wader/doc-example-usages 
doc,interp: Add some example usages to cli help
 2022-01-29 12:26:08 +01:00
Mattias Wadman
7bc2521938 doc,interp: Add some example usages to cli help 2022-01-29 12:21:51 +01:00
Mattias Wadman
8e47fb1a85 doc,matroska: Fix filesname in example 2022-01-29 12:20:47 +01:00
Mattias Wadman
a8664ed5ba doc: Add per format documentation 2022-01-29 12:01:36 +01:00
Mattias Wadman
fc0aacb654 interp: Cleanup display aliases, now: d, da, dd, dv, ddv 
Think it makes sense to have them all start with d.
Also f is often used as function argument name.
 2022-01-28 18:25:38 +01:00
Mattias Wadman
c52a1a2364 doc: Use f($a; $b) instead of jq f/2 notation 
Also some general polish
 2022-01-27 09:55:45 +01:00
Mattias Wadman
e792598c69 interp: Add grep_by/1 to recursively match using a filter 
Refactor other grep functions to use it
 2022-01-24 18:26:25 +01:00
Xentripetal
ee184075b7 Parse header using avro decoders. Still not certain this is the best idea. Will get opinions before finalizing. 2022-01-19 21:00:44 -06:00
Xentripetal
6e1f338ac3 Pull latest from wader 2022-01-19 19:39:12 -06:00
Xentripetal
06085a26f6 Undo change to doc/file.mp4. I have no idea how this got changed in the first place? Maybe some macos shenanigans. 2022-01-19 19:07:08 -06:00
Mattias Wadman
059e287227 todo: Cleanup add some known issues and deoder ideas 2022-01-18 17:52:51 +01:00
Mattias Wadman
6ca4767e24 doc: Improve formats graph a bit 
Force all dep edges to go from east to north
 2022-01-16 20:12:40 +01:00
Mattias Wadman
8e9700d0bd doc: Improve readme a bit and add torepr example 2022-01-15 19:26:58 +01:00
Xentripetal
788fd0258f Pull latest from wader 2022-01-13 20:59:56 -06:00
Mattias Wadman
8092151e52
Merge pull request #77 from wader/formats-sorts 
formats: Sort and make lists less likely to cause collision
 2022-01-12 18:42:05 +01:00
Mattias Wadman
729a6cae1e formats: Sort and make lists less likely to cause collision 2022-01-12 18:35:28 +01:00
Mattias Wadman
59b88034d4 doc: Document dev dependencies and related PRs/issues etc 2022-01-12 18:29:33 +01:00
Mattias Wadman
0b0f28e966 cbor: Add decoder 
Does not decode sematic tag types
Also fixes broken float16 support

Fixes #71
 2022-01-12 17:56:03 +01:00
Mattias Wadman
149cb3f45a interp: Add torepr/0 that converts decode value into what it reptresents 
Ex: fq -d msgpack torepr file.msgpack
Willoutput the JSON representation of the msgpack

Make per format *_torepr functions internal
 2022-01-12 17:33:58 +01:00
Mattias Wadman
809210bea6 doc: Add more dev tips 2022-01-11 12:28:21 +01:00
Mattias Wadman
a6bf62ce19 msgpack: Add decoder 
Fixes #25
 2022-01-10 19:33:09 +01:00
Mattias Wadman
af8e7efc61 bencode: Add decoder 
Closes #63
 2022-01-10 10:52:14 +01:00
Mattias Wadman
a0ccde99a5
Merge pull request #65 from wader/doc-dev-help 
doc,dev: Add some more decoder implementation help
 2022-01-09 17:03:42 +01:00
Mattias Wadman
b4694b6b9b doc,dev: Add some more decoder implementation help 2022-01-09 16:57:11 +01:00
Mattias Wadman
b3a09805a4 interp: Document bit opts funcs and add some error tests 2022-01-09 10:43:07 +01:00
Mattias Wadman
2c3e41111d doc,make: Add release script 2022-01-06 09:57:47 +01:00
Mattias Wadman
628f0f403d bson: Add decoder 
Only supports basic types for now
 2022-01-06 00:51:37 +01:00
Mattias Wadman
681dbc26b7 elf,ar: Add ar decoder, improved elf decoder 
Add x86, 386, arm64, arm v6, arm v7 linux elf test files
 2022-01-05 22:05:09 +01:00
Mattias Wadman
e365f22084 interp: Cleanup stdio usage and functions 2022-01-05 21:41:11 +01:00
Mattias Wadman
966863cb47 doc,dev: Add note how to setup windows dev env 2022-01-04 22:14:57 +01:00
Mattias Wadman
a41f0d4b5f windows: Correct @builtin include path join 
Failed because go embed paths always uses /
Switch to always use forward slash paths internally

Related to #21
 2022-01-04 22:10:53 +01:00
Mattias Wadman
b9aef39ed4 pcap,pcapng,bsd_loopback_frame: Add decoder, refactor link frame into a group 
Also fix incorrect struct name s/UDPDatagramIn/UDPPayloadIn/
 2022-01-03 21:06:27 +01:00
Mathieu Aubin
30aa0925cf
typ and links 2022-01-01 05:45:03 -05:00
Mattias Wadman
d440c4fa11
Merge pull request #45 from wader/doc-improve2 
doc: Some more dev infomration
 2021-12-31 17:18:24 +01:00
Mattias Wadman
6db7218093 doc: Some more dev infomration 2021-12-31 17:13:16 +01:00
Mathieu Aubin
c90e3e1d0c
typpp 2021-12-30 08:56:50 -05:00
Xentripetal
27789f2da5 Regenerate docs 2021-12-29 16:30:55 -06:00
Mattias Wadman
63485efda8 doc: Add hopes, pronounce and some more examples 2021-12-28 23:13:23 +01:00
Mattias Wadman
eb4a6fdbd6 doc: Remove empty comment 2021-12-22 19:34:06 +01:00
Mattias Wadman
6491d08fb0 doc: Add more common usages 2021-12-22 19:33:21 +01:00
Mattias Wadman
916cb30618 doc: Improve usage examples as bit 2021-12-20 17:21:07 +01:00
Mattias Wadman
d02c7c42ff doc: Add some more usage examples 2021-12-20 15:15:43 +01:00
Mattias Wadman
e1e8a236e6 doc: Add color/unicode section and move config section 2021-12-19 11:51:24 +01:00
Mattias Wadman
f7c7801bbe interp: Rework buffer regex support 
Now all regexp functions should work similar to string version for buffers.
Added scan_toend that works as scan but outputs buffer from match to end of buffer,
useful when trying to decode a format and you dont know or care to limit.
 2021-12-17 16:23:44 +01:00
Mattias Wadman
6ed2e2e72e interp: dump: Indicate arrays using jq-syntax 
Related to #16
 2021-12-08 16:24:28 +01:00
Mattias Wadman
9aec91a675 interp: match: Fix issue with regexp meta characters when matching using a buffer 2021-12-04 17:24:43 +01:00
Mattias Wadman
be46d5f4c1 doc: Cleanup todo and add some dev notes 2021-12-04 17:23:34 +01:00
Mattias Wadman
a7a58c8fb2 fq: Rename main.go to fq.go 2021-12-03 10:35:52 +01:00
Mattias Wadman
f55b1af6ac inet: Add tcp and ipv4 reassembly 
Also add tcp_stream and udp_payload to decode content
 2021-11-29 18:42:18 +01:00
Mattias Wadman
2f9d93d0ce doc: Improved readme and cleanup todo 2021-11-29 13:34:45 +01:00
Mattias Wadman
1f61704ad5 doc: Regenerate svg after ansisvg monospace update 2021-11-25 15:02:15 +01:00
Mattias Wadman
7b7faaf02b pcap: Add pcap, pcapng, ether8023, ipv4, udp, udp 2021-11-24 18:13:00 +01:00
Mattias Wadman
d6d3265b34 doc: Update README a bit 2021-11-22 14:16:59 +01:00
Mattias Wadman
df5bd19c75 doc: Update TODO 2021-11-21 22:13:26 +01:00
Mattias Wadman
1047d909ac doc: Document io packages a bit 2021-11-21 21:55:53 +01:00
Mattias Wadman
0480a2f722 decode: Some format decode and sub buffer work 2021-11-20 18:56:59 +01:00
Mattias Wadman
07b421011f interp: add topath/0 and make todescription return null if there is none 
Move out more code from funcs.jq
 2021-11-19 11:44:23 +01:00
Mattias Wadman
07c7daaf92 doc: Some rewording 2021-11-19 00:23:04 +01:00
Mattias Wadman
d838d2f613 zip: Add format decoder 2021-11-19 00:22:26 +01:00
Mattias Wadman
986d5ecc50 decode: Move registry package to decode/registry and add a format group type 2021-11-17 16:59:56 +01:00
Mattias Wadman
f9f866000e interp,decode: Add force option to ignore asserts 2021-11-16 13:03:56 +01:00
Mattias Wadman
3fafee8559 interp: Clearer help for -d 2021-11-16 08:43:29 +01:00