1
1
mirror of https://github.com/wader/fq.git synced 2024-12-23 21:31:33 +03:00
Commit Graph

71 Commits

Author SHA1 Message Date
Mattias Wadman
2fc16ae22a doc: Add some padding margin to formats table to make it less likely to cause git conflicts 2022-12-05 12:25:00 +01:00
@0xb17bea125
c4e7fc7919 tzif: moved document to tzif.md 2022-12-04 21:04:13 +09:00
@0xb17bea125
2ee6360b97 support tzif (time zone information format) 2022-12-04 21:02:39 +09:00
Mattias Wadman
ef2d5232ba mp4: Add png mapping
Also support samples decode for tracks without hdlr box
2022-11-25 19:49:18 +01:00
Mattias Wadman
c93301fc70 raw,bits,bytes: Replace raw format with bits and bytes format that decode to a binary
raw format was a hack to skip decoding to be able to get a binary using tobyte etc.
Now you can do fq -d bytes ... instead of fq -d raw 'tobytes | ...'
2022-11-20 19:51:12 +01:00
Mattias Wadman
0cf46e1196 wav,avi,avc_au: Add avi decoder and refactor wav decoder
avi and wav now share  common riff decode code
avc_au decoder can now handle annexb format
2022-11-19 13:11:27 +01:00
Mattias Wadman
c3a0686c9e mp3_frame_tags: Refactor and rename xing format to mp3_frame_tags
Make more sense to use a more general name when there is both info and xing
and possibly vbri in the future. Also both xing and info seems to have extension
fields

Remove some old tests that are not really needed anymore.
2022-11-16 17:32:03 +01:00
Mattias Wadman
2a86d323b4 doc,rtmp,pcap,markdown: Add more examples 2022-10-09 19:18:14 +02:00
Mattias Wadman
09ea08f3aa
Merge pull request #427 from dgmcdona/bplist
Adds support for Apple Binary Plist, version 00
2022-10-04 14:25:53 +02:00
Mattias Wadman
0fdc03a4f1 doc: Fix some incorrect example prompts 2022-09-27 13:42:31 +02:00
David McDonald
bcccde2358 Fixes and embeds documentation 2022-09-24 12:57:11 -05:00
David McDonald
a77cec921f Added documentation and tests, fixed bad date parsing 2022-09-23 20:23:23 -05:00
Mattias Wadman
a7a101ca67 doc,help: Nicer format help and move help tests into each format
Also add authors to avro and macho decoders

Generated with:
for i in $(go run . -r -n '_registry | [([([.files[][].name | split(".")[0]] | unique[]), (.formats[].name)] | count[] | select(.[1] == 2)[0]), (.formats[] | select(.decode_in_arg).name)] | unique[]'); do DIR=$(dirname $(find format -name $i.go)); echo "$ fq -h $i" > "$DIR/testdata/help_$i.fqtest" ; done
2022-09-22 23:07:58 +02:00
Mattias Wadman
725ab1b17b doc,html,xml: Add more documentation and examples 2022-09-20 17:35:27 +02:00
Mattias Wadman
fe64530e11 csv: Add tsv and header example
Also fix string esacpe in fuzz options parser
2022-09-15 01:16:45 +02:00
Mattias Wadman
1385b5d0ba wasm: Add some documentation 2022-09-12 11:53:48 +02:00
Mattias Wadman
abd19ed89b doc: Fix format sections a bit 2022-09-11 09:55:56 +02:00
Mattias Wadman
87b2c6c10c help,doc: Use markdown for format documentation again
Markdown is used as is in online documentation and in cli the markdown decoder
is used to decode and the some jq code massages it into something cli friendly.

Was just too much of a mess to have doc in jq.
2022-09-11 00:53:29 +02:00
Mattias Wadman
095e1161b7 xml: Switch from "-" to "@" as attribute prefix and make it an option
Seems to be more common.
2022-08-25 17:15:23 +02:00
Mattias Wadman
8fc43533a9 prores_frame: Add basic container and frame header decoder 2022-08-24 19:33:35 +02:00
@0xb17bea125
ce43887298
wasm: make doc 2022-08-20 14:58:05 +09:00
Mattias Wadman
701c67c136 jsonl: Add decoder, also tojsonl encoder 2022-08-15 19:43:59 +02:00
Mattias Wadman
46dca8cdc5 dns: Don't use dns (udp) format for tcp also
Fixes issue with TCP streams being probed as dns over TCP
2022-08-09 18:15:34 +02:00
Mattias Wadman
725c8e83ab macho: Split into macho/macho_fat, fix offset issue and add string decoding
Split fat macho into own decoder macho_fat. This also fixes issue with section
offset etc not being correct as they are from the start of each embedded file.

Make all address and offset field be in hex.

Decode __cstring, __ustring and __cfstring sections.

Fix LC_ENCRYPTION_INFO_64 missing pading issue.

Skip ranging for __bss and __common as they dont have any data in the file.

Simplifed magic handling a bit and add symbols.

Simplified state struct field, had redudant struct.
2022-08-02 14:36:53 +02:00
Mattias Wadman
cae288e6be format,intepr: Refactor json, yaml, etc into formats also move out related functions
json, yaml, toml, xml, html, csv are now normal formats and most of them also particiate
in probing (not html and csv).

Also fixes a bunch of bugs in to/fromxml, to/fromjq etc.
2022-07-23 21:48:45 +02:00
Mattias Wadman
64f3e5c717 fairplay: Add basic SPC decoder and PSSH system id 2022-07-07 17:11:55 +02:00
Mattias Wadman
417255b72b bitcoin: Add blkdat, block, transcation and script decoder 2022-07-04 09:59:51 +02:00
Mattias Wadman
0863374f8c doc: Correct bencode spec URL
Also rename _tojq value_sep to object_sep and fix formats_diagram.jq to use tomd5
2022-06-13 18:49:34 +02:00
Mattias Wadman
aa694e3f20 zip: s/Decompress/Uncompress/ 2022-05-26 19:36:06 +02:00
Mattias Wadman
47350e46a6 zip: Add uncompress=false test and some docs 2022-05-26 16:39:01 +02:00
Mattias Wadman
8021492197 interp: help: Fix incorrect options example 2022-05-26 12:52:51 +02:00
Mattias Wadman
e641274487 zip: Add zip64 support and uncompress option
{uncompress:boolean} uncompress and probe, default true
2022-05-26 12:46:50 +02:00
Mattias Wadman
1aaaefb0ac wav,bencode,mpeg_ps_packet,id3v1: Random fixes
wav: Rest of file should description not symbolic value
bencode: Remove redundant torepr example
mpeg_ps_packet,id3v1: Remove whitespace in symbolic value
2022-05-05 19:45:05 +02:00
Mattias Wadman
f4480c6fe5 decode,interp: Support for format specific options
interp: Refactor format help and also include options
interp: Add -o name=@path to load file content as value (not documented yet, might change)
interp,decode: Expose decode out value as _out (might change)
interp: Refactor foramts.jq into format_{decode,func,include}.jq
interp: Refactor torepr into _format_func for generic format function overloading
interp: Refactor -o options parsing to be more generic and collect unknowns options to be used as format options
decode of decode alises
func for format overloaded functions
include for format specific jq functions (also _help, torepr etc)
flac_frame: Add bits_per_sample option
mp3: Add max_unique_header_config and max_sync_seek options
mp4: Add decode_samples and allow_truncate options
avc_au: Has length_size option
hevc_au: Has length_size option
aac_frame: Has object_typee option
doc: Rewrite format doc generation, less hack more jq
2022-05-01 17:08:30 +02:00
Mattias Wadman
21ad628add interp: dump: Show field name for compound values in arrays
mp3: Remove one level of struct for granales
dump: Makes it a bit easier to understand array of arrays/structs.
avro_ocf: Rename ocf.go -> avro_ocr.go to have same name of source file as format
pcap,pcapng: Rename tcp flow struct to tcp_connection, makes more sense i think
mp4: Add ISOBMFF to format description
2022-04-21 17:47:15 +02:00
Mattias Wadman
788b0ac197 rtmp,amf0: Improve decoders, aac asc, chunk stream interrupt, fix amf0 ecma arrays
rtmp: Handle chunk streams that have been interrupted (capture terminated in middle of stream etc)
rtmp: Decode AAC ASC
rtmp: Add ffmpeg client/server stream test
rtmp: Decode user control messages
rtmo: Decode all data messages
amf0: Fix ecma array decoding
2022-04-04 14:42:16 +02:00
Mattias Wadman
e8dc7112b6 ipv6,icmpv6: Add decoder
Refactor to use groups between network layers to make them less coupled and reusable:
link_frame (ethernet etc)
inet_packet (ipv4,ipv6 etc)
ip_packet (tcp packet (not stream), udp segment etc)

Rename data to payload as i think it makes more sense for network data
2022-04-03 17:43:51 +02:00
Mattias Wadman
d8aaf30345 rtmp,amf0: Add decoders
Basic RTMP support, used via tcp_stream group (pcap etc) or manually.
Basic AMF0 support, mostly what is used in RTMP.
2022-04-01 13:07:20 +02:00
Mattias Wadman
74978c9d08 hevc: Add hevc_vps, hevc_sps and hevc_pps decoders
Some extensions and other things still missing
2022-03-24 18:56:56 +01:00
Mattias Wadman
3f50bb90a6 interp: Rework formats and options help
Replace --formats with --help formats
Add --help options
2022-03-03 18:25:20 +01:00
Mattias Wadman
920629f5fd doc: Regenerate and fix macho section size
Make macho format documentaion a bit more consistent.
2022-03-03 09:55:53 +01:00
Mattias Wadman
758b2d0e34 doc: Regenerate after macho merge 2022-02-28 10:28:21 +01:00
siddik.acil
2e7767cd05 macho: remake docs 2022-02-21 20:33:30 +01:00
Mattias Wadman
d334c2d4ea doc: Add href in supported format list 2022-02-19 01:33:45 +01:00
Mattias Wadman
c95b0d6d19 doc: Forgot make doc 2022-02-16 20:13:28 +01:00
Xentripetal
0300c955c5 Take heading off doc to match make doc format 2022-02-09 20:59:29 -06:00
Xentripetal
2605bce4e3 Lint and add basic doc 2022-02-09 20:46:12 -06:00
Xentripetal
5b11385ea4 Pull latest from wader 2022-02-07 17:35:33 -06:00
Mattias Wadman
0312c92c31 asn1_ber: Add more doc and multiple outputs for frompem 2022-02-07 17:41:05 +01:00
Mattias Wadman
0829c167cc asn1_ber: Add decoder
Basic support, can also do CER and DER but without any extra validation.
No schema support.

Redo format doc.md usage a bit, now format/<dir>/<format>.md instead.

Related to #20
2022-02-06 22:13:03 +01:00