Mattias Wadman
47350e46a6
zip: Add uncompress=false test and some docs
2022-05-26 16:39:01 +02:00
Mattias Wadman
8021492197
interp: help: Fix incorrect options example
2022-05-26 12:52:51 +02:00
Mattias Wadman
e641274487
zip: Add zip64 support and uncompress option
...
{uncompress:boolean} uncompress and probe, default true
2022-05-26 12:46:50 +02:00
Mattias Wadman
a980656c15
doc: go 1.18 and improve intro text a bit
2022-05-23 21:57:45 +02:00
Mattias Wadman
e9d9f8aef9
fq: Use go 1.18
...
Rename s/interface{}/any/g
Preparation for using generics in decode API and native jq funcations etc
Remove some unused linter ignores as linter has been fixed
2022-05-20 15:23:16 +02:00
Mattias Wadman
1aaaefb0ac
wav,bencode,mpeg_ps_packet,id3v1: Random fixes
...
wav: Rest of file should description not symbolic value
bencode: Remove redundant torepr example
mpeg_ps_packet,id3v1: Remove whitespace in symbolic value
2022-05-05 19:45:05 +02:00
Mattias Wadman
f4480c6fe5
decode,interp: Support for format specific options
...
interp: Refactor format help and also include options
interp: Add -o name=@path to load file content as value (not documented yet, might change)
interp,decode: Expose decode out value as _out (might change)
interp: Refactor foramts.jq into format_{decode,func,include}.jq
interp: Refactor torepr into _format_func for generic format function overloading
interp: Refactor -o options parsing to be more generic and collect unknowns options to be used as format options
decode of decode alises
func for format overloaded functions
include for format specific jq functions (also _help, torepr etc)
flac_frame: Add bits_per_sample option
mp3: Add max_unique_header_config and max_sync_seek options
mp4: Add decode_samples and allow_truncate options
avc_au: Has length_size option
hevc_au: Has length_size option
aac_frame: Has object_typee option
doc: Rewrite format doc generation, less hack more jq
2022-05-01 17:08:30 +02:00
Mattias Wadman
21ad628add
interp: dump: Show field name for compound values in arrays
...
mp3: Remove one level of struct for granales
dump: Makes it a bit easier to understand array of arrays/structs.
avro_ocf: Rename ocf.go -> avro_ocr.go to have same name of source file as format
pcap,pcapng: Rename tcp flow struct to tcp_connection, makes more sense i think
mp4: Add ISOBMFF to format description
2022-04-21 17:47:15 +02:00
Mattias Wadman
788b0ac197
rtmp,amf0: Improve decoders, aac asc, chunk stream interrupt, fix amf0 ecma arrays
...
rtmp: Handle chunk streams that have been interrupted (capture terminated in middle of stream etc)
rtmp: Decode AAC ASC
rtmp: Add ffmpeg client/server stream test
rtmp: Decode user control messages
rtmo: Decode all data messages
amf0: Fix ecma array decoding
2022-04-04 14:42:16 +02:00
Mattias Wadman
e8dc7112b6
ipv6,icmpv6: Add decoder
...
Refactor to use groups between network layers to make them less coupled and reusable:
link_frame (ethernet etc)
inet_packet (ipv4,ipv6 etc)
ip_packet (tcp packet (not stream), udp segment etc)
Rename data to payload as i think it makes more sense for network data
2022-04-03 17:43:51 +02:00
Mattias Wadman
d8aaf30345
rtmp,amf0: Add decoders
...
Basic RTMP support, used via tcp_stream group (pcap etc) or manually.
Basic AMF0 support, mostly what is used in RTMP.
2022-04-01 13:07:20 +02:00
Mattias Wadman
74978c9d08
hevc: Add hevc_vps, hevc_sps and hevc_pps decoders
...
Some extensions and other things still missing
2022-03-24 18:56:56 +01:00
Mattias Wadman
7cea8757b1
doc: Add 0.0.6 changes
2022-03-11 17:01:26 +01:00
Mattias Wadman
dff7e7dadb
interp: Cleanup binary regexp overloading and add explode
...
Add explode for binary that explode into an array of the binary unit tobits
Remove scan_toend as it feels a bit weird and one can use tobytes | .[match(...):] instead
Some general cleanup also
2022-03-09 20:17:12 +01:00
Mattias Wadman
b97776c9e3
doc: Add fq bts2022 presentation
2022-03-08 18:20:18 +01:00
Mattias Wadman
3f50bb90a6
interp: Rework formats and options help
...
Replace --formats with --help formats
Add --help options
2022-03-03 18:25:20 +01:00
Mattias Wadman
920629f5fd
doc: Regenerate and fix macho section size
...
Make macho format documentaion a bit more consistent.
2022-03-03 09:55:53 +01:00
Mattias Wadman
0a043f9096
repl,interp: Refactor repl and slurp
...
Now repl, slurp and help implemented using same query rewrite.
Include filename context in error if possible.
Add spew function that does opposite of slurp.
Start of help infra, not done or documented yet.
Show error pointer on parse error.
Rename internal eval to _eval and make eval be wrapper that
does rewrite and has various eror handling etc.
Nicer repl, slupr and help errors.
2022-03-01 18:31:00 +01:00
Mattias Wadman
d3397cf9e7
doc: Tweak format diagram
2022-02-28 10:42:39 +01:00
Mattias Wadman
758b2d0e34
doc: Regenerate after macho merge
2022-02-28 10:28:21 +01:00
siddik.acil
2e7767cd05
macho: remake docs
2022-02-21 20:33:30 +01:00
Mattias Wadman
d334c2d4ea
doc: Add href in supported format list
2022-02-19 01:33:45 +01:00
Mattias Wadman
c95b0d6d19
doc: Forgot make doc
2022-02-16 20:13:28 +01:00
Mattias Wadman
68ba2e810e
doc: Add CHANGES.md documenting release changes
2022-02-13 23:36:11 +01:00
Mattias Wadman
d4cc544448
Merge pull request #153 from wader/doc-dev-more-decode-api
...
doc: More decode API details and polish
2022-02-13 17:22:04 +01:00
Mattias Wadman
b3504680f7
doc: More decode API details and polish
2022-02-13 16:28:59 +01:00
Mattias Wadman
dd3e40fb1c
doc: Unbreak formats_digaram.jq since radix change
2022-02-13 10:31:53 +01:00
Xentripetal
d54011663f
make doc
2022-02-12 22:17:32 -06:00
Xentripetal
9636613ec6
Merge branch 'master' of https://github.com/wader/fq into avro
2022-02-12 22:16:50 -06:00
Mattias Wadman
b86da7aecc
doc: Add inital decoder API documentation
2022-02-12 16:13:29 +01:00
Mattias Wadman
48a19cb82c
interp: Add paste function to allow pasting text into REPL etc
...
Also refactor readline and eval args into option struct and partinally start
addressing some side effects during completion.
2022-02-11 17:59:06 +01:00
Xentripetal
0300c955c5
Take heading off doc to match make doc format
2022-02-09 20:59:29 -06:00
Xentripetal
2605bce4e3
Lint and add basic doc
2022-02-09 20:46:12 -06:00
Mattias Wadman
36307857de
doc: Cleanup and add more decode value and binary documentation
2022-02-09 21:08:01 +01:00
Mattias Wadman
06245d1295
binary,decode,doc: Rename buffer to binary and add some documentation
...
Rename buffer to binary. Still some work left what to call buffer/binary in decode code.
Document decode value and binary type
Fix proper unit padding for tobytes and add still undocumenated extra padding argument.
Add some additional binary tests
2022-02-08 22:20:28 +01:00
Xentripetal
5b11385ea4
Pull latest from wader
2022-02-07 17:35:33 -06:00
Mattias Wadman
0312c92c31
asn1_ber: Add more doc and multiple outputs for frompem
2022-02-07 17:41:05 +01:00
Mattias Wadman
0829c167cc
asn1_ber: Add decoder
...
Basic support, can also do CER and DER but without any extra validation.
No schema support.
Redo format doc.md usage a bit, now format/<dir>/<format>.md instead.
Related to #20
2022-02-06 22:13:03 +01:00
Mattias Wadman
91b0fed929
Merge pull request #127 from wader/doc-foramts-option-links
...
doc: Add format links to format table
2022-02-05 18:24:03 +01:00
Mattias Wadman
c15f5283c0
doc: Add format links to format table
2022-02-05 18:15:18 +01:00
Mattias Wadman
366f6b185c
interp: Support force decode as -o force=true
2022-02-05 17:24:42 +01:00
Mattias Wadman
7c5215347d
bitio,decode: Refactor bitio usage and make buffer slicing more correct
...
Remove bitio.Buffer layer. bitio.Buffer was a kitchen sink layer with helpers
now it's just a buffer and most functions have been moved to decode instead.
bitio package now only have primitive types and functions simialar to standard
library io and bytes packages.
Make nearly eveything internally use bitio.Bit* interfaces so that slicing work
correctly this will also make it possible to start experimenting with more
complicated silcing helpers, ex things like:
breplace(.header.bitrate; 123) to get a new buffer with bitrate changed.
2022-02-04 21:41:53 +01:00
Mattias Wadman
710c29b24e
doc: Color edges in diagram based on dest
2022-02-02 19:39:33 +01:00
Mattias Wadman
2ab395a03b
protobuf: Add note about sub message decoding
...
Fix and cleanup some other doc also
2022-02-01 22:22:19 +01:00
Mattias Wadman
6b51b0676f
doc: More display alias leftover fixes
2022-01-29 13:22:10 +01:00
Mattias Wadman
f0ce7179b2
doc: Document display and some more jq hints
...
Also fixed some verbose -> dv leftovers
2022-01-29 13:10:25 +01:00
Mattias Wadman
bc20f09fdb
Merge pull request #117 from wader/make-cleanup-targets
...
make: Cleanup some not very used targets
2022-01-29 12:31:10 +01:00
Mattias Wadman
b66e3287f2
Merge pull request #112 from wader/interp-cleanup-d-alises
...
interp: Cleanup display aliases, now: d, da, dd, dv, ddv
2022-01-29 12:28:46 +01:00
Mattias Wadman
fc52f20ed1
Merge pull request #116 from wader/doc-fix-filenames
...
doc,matroska: Fix filesname in example
2022-01-29 12:27:38 +01:00
Mattias Wadman
bf170be8ea
make: Cleanup some not very used targets
2022-01-29 12:27:15 +01:00
Mattias Wadman
2efc3cba9b
Merge pull request #115 from wader/doc-example-usages
...
doc,interp: Add some example usages to cli help
2022-01-29 12:26:08 +01:00
Mattias Wadman
7bc2521938
doc,interp: Add some example usages to cli help
2022-01-29 12:21:51 +01:00
Mattias Wadman
8e47fb1a85
doc,matroska: Fix filesname in example
2022-01-29 12:20:47 +01:00
Mattias Wadman
a8664ed5ba
doc: Add per format documentation
2022-01-29 12:01:36 +01:00
Mattias Wadman
fc0aacb654
interp: Cleanup display aliases, now: d, da, dd, dv, ddv
...
Think it makes sense to have them all start with d.
Also f is often used as function argument name.
2022-01-28 18:25:38 +01:00
Mattias Wadman
c52a1a2364
doc: Use f($a; $b) instead of jq f/2 notation
...
Also some general polish
2022-01-27 09:55:45 +01:00
Mattias Wadman
e792598c69
interp: Add grep_by/1 to recursively match using a filter
...
Refactor other grep functions to use it
2022-01-24 18:26:25 +01:00
Xentripetal
ee184075b7
Parse header using avro decoders. Still not certain this is the best idea. Will get opinions before finalizing.
2022-01-19 21:00:44 -06:00
Xentripetal
6e1f338ac3
Pull latest from wader
2022-01-19 19:39:12 -06:00
Xentripetal
06085a26f6
Undo change to doc/file.mp4. I have no idea how this got changed in the first place? Maybe some macos shenanigans.
2022-01-19 19:07:08 -06:00
Mattias Wadman
059e287227
todo: Cleanup add some known issues and deoder ideas
2022-01-18 17:52:51 +01:00
Mattias Wadman
6ca4767e24
doc: Improve formats graph a bit
...
Force all dep edges to go from east to north
2022-01-16 20:12:40 +01:00
Mattias Wadman
8e9700d0bd
doc: Improve readme a bit and add torepr example
2022-01-15 19:26:58 +01:00
Xentripetal
788fd0258f
Pull latest from wader
2022-01-13 20:59:56 -06:00
Mattias Wadman
8092151e52
Merge pull request #77 from wader/formats-sorts
...
formats: Sort and make lists less likely to cause collision
2022-01-12 18:42:05 +01:00
Mattias Wadman
729a6cae1e
formats: Sort and make lists less likely to cause collision
2022-01-12 18:35:28 +01:00
Mattias Wadman
59b88034d4
doc: Document dev dependencies and related PRs/issues etc
2022-01-12 18:29:33 +01:00
Mattias Wadman
0b0f28e966
cbor: Add decoder
...
Does not decode sematic tag types
Also fixes broken float16 support
Fixes #71
2022-01-12 17:56:03 +01:00
Mattias Wadman
149cb3f45a
interp: Add torepr/0 that converts decode value into what it reptresents
...
Ex: fq -d msgpack torepr file.msgpack
Willoutput the JSON representation of the msgpack
Make per format *_torepr functions internal
2022-01-12 17:33:58 +01:00
Mattias Wadman
809210bea6
doc: Add more dev tips
2022-01-11 12:28:21 +01:00
Mattias Wadman
a6bf62ce19
msgpack: Add decoder
...
Fixes #25
2022-01-10 19:33:09 +01:00
Mattias Wadman
af8e7efc61
bencode: Add decoder
...
Closes #63
2022-01-10 10:52:14 +01:00
Mattias Wadman
a0ccde99a5
Merge pull request #65 from wader/doc-dev-help
...
doc,dev: Add some more decoder implementation help
2022-01-09 17:03:42 +01:00
Mattias Wadman
b4694b6b9b
doc,dev: Add some more decoder implementation help
2022-01-09 16:57:11 +01:00
Mattias Wadman
b3a09805a4
interp: Document bit opts funcs and add some error tests
2022-01-09 10:43:07 +01:00
Mattias Wadman
2c3e41111d
doc,make: Add release script
2022-01-06 09:57:47 +01:00
Mattias Wadman
628f0f403d
bson: Add decoder
...
Only supports basic types for now
2022-01-06 00:51:37 +01:00
Mattias Wadman
681dbc26b7
elf,ar: Add ar decoder, improved elf decoder
...
Add x86, 386, arm64, arm v6, arm v7 linux elf test files
2022-01-05 22:05:09 +01:00
Mattias Wadman
e365f22084
interp: Cleanup stdio usage and functions
2022-01-05 21:41:11 +01:00
Mattias Wadman
966863cb47
doc,dev: Add note how to setup windows dev env
2022-01-04 22:14:57 +01:00
Mattias Wadman
a41f0d4b5f
windows: Correct @builtin include path join
...
Failed because go embed paths always uses /
Switch to always use forward slash paths internally
Related to #21
2022-01-04 22:10:53 +01:00
Mattias Wadman
b9aef39ed4
pcap,pcapng,bsd_loopback_frame: Add decoder, refactor link frame into a group
...
Also fix incorrect struct name s/UDPDatagramIn/UDPPayloadIn/
2022-01-03 21:06:27 +01:00
Mathieu Aubin
30aa0925cf
typ and links
2022-01-01 05:45:03 -05:00
Mattias Wadman
d440c4fa11
Merge pull request #45 from wader/doc-improve2
...
doc: Some more dev infomration
2021-12-31 17:18:24 +01:00
Mattias Wadman
6db7218093
doc: Some more dev infomration
2021-12-31 17:13:16 +01:00
Mathieu Aubin
c90e3e1d0c
typpp
2021-12-30 08:56:50 -05:00
Xentripetal
27789f2da5
Regenerate docs
2021-12-29 16:30:55 -06:00
Mattias Wadman
63485efda8
doc: Add hopes, pronounce and some more examples
2021-12-28 23:13:23 +01:00
Mattias Wadman
eb4a6fdbd6
doc: Remove empty comment
2021-12-22 19:34:06 +01:00
Mattias Wadman
6491d08fb0
doc: Add more common usages
2021-12-22 19:33:21 +01:00
Mattias Wadman
916cb30618
doc: Improve usage examples as bit
2021-12-20 17:21:07 +01:00
Mattias Wadman
d02c7c42ff
doc: Add some more usage examples
2021-12-20 15:15:43 +01:00
Mattias Wadman
e1e8a236e6
doc: Add color/unicode section and move config section
2021-12-19 11:51:24 +01:00
Mattias Wadman
f7c7801bbe
interp: Rework buffer regex support
...
Now all regexp functions should work similar to string version for buffers.
Added scan_toend that works as scan but outputs buffer from match to end of buffer,
useful when trying to decode a format and you dont know or care to limit.
2021-12-17 16:23:44 +01:00
Mattias Wadman
6ed2e2e72e
interp: dump: Indicate arrays using jq-syntax
...
Related to #16
2021-12-08 16:24:28 +01:00
Mattias Wadman
9aec91a675
interp: match: Fix issue with regexp meta characters when matching using a buffer
2021-12-04 17:24:43 +01:00
Mattias Wadman
be46d5f4c1
doc: Cleanup todo and add some dev notes
2021-12-04 17:23:34 +01:00
Mattias Wadman
a7a58c8fb2
fq: Rename main.go to fq.go
2021-12-03 10:35:52 +01:00
Mattias Wadman
f55b1af6ac
inet: Add tcp and ipv4 reassembly
...
Also add tcp_stream and udp_payload to decode content
2021-11-29 18:42:18 +01:00
Mattias Wadman
2f9d93d0ce
doc: Improved readme and cleanup todo
2021-11-29 13:34:45 +01:00
