1
1
mirror of https://github.com/wader/fq.git synced 2024-11-27 14:14:58 +03:00
Commit Graph

85 Commits

Author SHA1 Message Date
Mattias Wadman
b60aceca9e matroska: Add decode_samples option
Also change the option help a bit, maybe medid and support is confusing, we
might evetually decode non-audio/video things.
2023-02-08 11:31:41 +01:00
Mattias Wadman
2d82c05f64 mp3: Add max_unknown option to fail decode if too much unknown bits
Hopefully help fix even more miss-detections
2023-01-25 15:27:31 +01:00
Mattias Wadman
e3ae1440c9 interp: Rename to/from<format> functions to to_/from_<format>
Feels less cluttered, easier to read and more consistent.

Still keep tovalue, tobytes etc that are more basic functions this
only renamed format related functions.
Also there is an exceptin for to/fromjson as it comes from jq.

Also fixes lots of spelling errors while reading thru.
2022-12-21 17:48:39 +01:00
David McDonald
22064f50d6 doc: remake 2022-12-19 15:27:48 -06:00
Mattias Wadman
1dea40e6af mp4,doc: Add JSON box tree example and reorder a bit 2022-12-15 14:15:31 +01:00
Mattias Wadman
9e5a072e50 mp3_frame_tags: Covert to decode group and split to mp3_frame_{xing,vbri} decoders 2022-12-15 12:10:06 +01:00
David McDonald
4aad2fdeea doc: remake 2022-12-05 22:03:26 -06:00
David McDonald
34c560c2f5 Merge branch 'master' of https://github.com/wader/fq into macos_bookmark 2022-12-05 22:01:44 -06:00
Mattias Wadman
2fc16ae22a doc: Add some padding margin to formats table to make it less likely to cause git conflicts 2022-12-05 12:25:00 +01:00
@0xb17bea125
c4e7fc7919 tzif: moved document to tzif.md 2022-12-04 21:04:13 +09:00
@0xb17bea125
2ee6360b97 support tzif (time zone information format) 2022-12-04 21:02:39 +09:00
David McDonald
b872b1a337 doc: remake 2022-12-03 17:36:56 -06:00
David McDonald
f5e25fca26 updated docs 2022-12-01 23:31:57 -06:00
David McDonald
44c91d8296 tweaks apple_bookmark markdown documentation 2022-12-01 23:31:57 -06:00
David McDonald
4a28e44f44 changes decoder package name from bookmark to apple_bookmark 2022-12-01 23:31:57 -06:00
David McDonald
0a72635a78 remade documentation 2022-12-01 23:31:57 -06:00
David McDonald
34db9d7fea regenerated docs, added tests, fixed torepr 2022-12-01 23:31:57 -06:00
Mattias Wadman
ef2d5232ba mp4: Add png mapping
Also support samples decode for tracks without hdlr box
2022-11-25 19:49:18 +01:00
Mattias Wadman
c93301fc70 raw,bits,bytes: Replace raw format with bits and bytes format that decode to a binary
raw format was a hack to skip decoding to be able to get a binary using tobyte etc.
Now you can do fq -d bytes ... instead of fq -d raw 'tobytes | ...'
2022-11-20 19:51:12 +01:00
Mattias Wadman
0cf46e1196 wav,avi,avc_au: Add avi decoder and refactor wav decoder
avi and wav now share  common riff decode code
avc_au decoder can now handle annexb format
2022-11-19 13:11:27 +01:00
Mattias Wadman
c3a0686c9e mp3_frame_tags: Refactor and rename xing format to mp3_frame_tags
Make more sense to use a more general name when there is both info and xing
and possibly vbri in the future. Also both xing and info seems to have extension
fields

Remove some old tests that are not really needed anymore.
2022-11-16 17:32:03 +01:00
Mattias Wadman
2a86d323b4 doc,rtmp,pcap,markdown: Add more examples 2022-10-09 19:18:14 +02:00
Mattias Wadman
09ea08f3aa
Merge pull request #427 from dgmcdona/bplist
Adds support for Apple Binary Plist, version 00
2022-10-04 14:25:53 +02:00
Mattias Wadman
0fdc03a4f1 doc: Fix some incorrect example prompts 2022-09-27 13:42:31 +02:00
David McDonald
bcccde2358 Fixes and embeds documentation 2022-09-24 12:57:11 -05:00
David McDonald
a77cec921f Added documentation and tests, fixed bad date parsing 2022-09-23 20:23:23 -05:00
Mattias Wadman
a7a101ca67 doc,help: Nicer format help and move help tests into each format
Also add authors to avro and macho decoders

Generated with:
for i in $(go run . -r -n '_registry | [([([.files[][].name | split(".")[0]] | unique[]), (.formats[].name)] | count[] | select(.[1] == 2)[0]), (.formats[] | select(.decode_in_arg).name)] | unique[]'); do DIR=$(dirname $(find format -name $i.go)); echo "$ fq -h $i" > "$DIR/testdata/help_$i.fqtest" ; done
2022-09-22 23:07:58 +02:00
Mattias Wadman
725ab1b17b doc,html,xml: Add more documentation and examples 2022-09-20 17:35:27 +02:00
Mattias Wadman
fe64530e11 csv: Add tsv and header example
Also fix string esacpe in fuzz options parser
2022-09-15 01:16:45 +02:00
Mattias Wadman
1385b5d0ba wasm: Add some documentation 2022-09-12 11:53:48 +02:00
Mattias Wadman
abd19ed89b doc: Fix format sections a bit 2022-09-11 09:55:56 +02:00
Mattias Wadman
87b2c6c10c help,doc: Use markdown for format documentation again
Markdown is used as is in online documentation and in cli the markdown decoder
is used to decode and the some jq code massages it into something cli friendly.

Was just too much of a mess to have doc in jq.
2022-09-11 00:53:29 +02:00
Mattias Wadman
095e1161b7 xml: Switch from "-" to "@" as attribute prefix and make it an option
Seems to be more common.
2022-08-25 17:15:23 +02:00
Mattias Wadman
8fc43533a9 prores_frame: Add basic container and frame header decoder 2022-08-24 19:33:35 +02:00
@0xb17bea125
ce43887298
wasm: make doc 2022-08-20 14:58:05 +09:00
Mattias Wadman
701c67c136 jsonl: Add decoder, also tojsonl encoder 2022-08-15 19:43:59 +02:00
Mattias Wadman
46dca8cdc5 dns: Don't use dns (udp) format for tcp also
Fixes issue with TCP streams being probed as dns over TCP
2022-08-09 18:15:34 +02:00
Mattias Wadman
725c8e83ab macho: Split into macho/macho_fat, fix offset issue and add string decoding
Split fat macho into own decoder macho_fat. This also fixes issue with section
offset etc not being correct as they are from the start of each embedded file.

Make all address and offset field be in hex.

Decode __cstring, __ustring and __cfstring sections.

Fix LC_ENCRYPTION_INFO_64 missing pading issue.

Skip ranging for __bss and __common as they dont have any data in the file.

Simplifed magic handling a bit and add symbols.

Simplified state struct field, had redudant struct.
2022-08-02 14:36:53 +02:00
Mattias Wadman
cae288e6be format,intepr: Refactor json, yaml, etc into formats also move out related functions
json, yaml, toml, xml, html, csv are now normal formats and most of them also particiate
in probing (not html and csv).

Also fixes a bunch of bugs in to/fromxml, to/fromjq etc.
2022-07-23 21:48:45 +02:00
Mattias Wadman
64f3e5c717 fairplay: Add basic SPC decoder and PSSH system id 2022-07-07 17:11:55 +02:00
Mattias Wadman
417255b72b bitcoin: Add blkdat, block, transcation and script decoder 2022-07-04 09:59:51 +02:00
Mattias Wadman
0863374f8c doc: Correct bencode spec URL
Also rename _tojq value_sep to object_sep and fix formats_diagram.jq to use tomd5
2022-06-13 18:49:34 +02:00
Mattias Wadman
aa694e3f20 zip: s/Decompress/Uncompress/ 2022-05-26 19:36:06 +02:00
Mattias Wadman
47350e46a6 zip: Add uncompress=false test and some docs 2022-05-26 16:39:01 +02:00
Mattias Wadman
8021492197 interp: help: Fix incorrect options example 2022-05-26 12:52:51 +02:00
Mattias Wadman
e641274487 zip: Add zip64 support and uncompress option
{uncompress:boolean} uncompress and probe, default true
2022-05-26 12:46:50 +02:00
Mattias Wadman
1aaaefb0ac wav,bencode,mpeg_ps_packet,id3v1: Random fixes
wav: Rest of file should description not symbolic value
bencode: Remove redundant torepr example
mpeg_ps_packet,id3v1: Remove whitespace in symbolic value
2022-05-05 19:45:05 +02:00
Mattias Wadman
f4480c6fe5 decode,interp: Support for format specific options
interp: Refactor format help and also include options
interp: Add -o name=@path to load file content as value (not documented yet, might change)
interp,decode: Expose decode out value as _out (might change)
interp: Refactor foramts.jq into format_{decode,func,include}.jq
interp: Refactor torepr into _format_func for generic format function overloading
interp: Refactor -o options parsing to be more generic and collect unknowns options to be used as format options
decode of decode alises
func for format overloaded functions
include for format specific jq functions (also _help, torepr etc)
flac_frame: Add bits_per_sample option
mp3: Add max_unique_header_config and max_sync_seek options
mp4: Add decode_samples and allow_truncate options
avc_au: Has length_size option
hevc_au: Has length_size option
aac_frame: Has object_typee option
doc: Rewrite format doc generation, less hack more jq
2022-05-01 17:08:30 +02:00
Mattias Wadman
21ad628add interp: dump: Show field name for compound values in arrays
mp3: Remove one level of struct for granales
dump: Makes it a bit easier to understand array of arrays/structs.
avro_ocf: Rename ocf.go -> avro_ocr.go to have same name of source file as format
pcap,pcapng: Rename tcp flow struct to tcp_connection, makes more sense i think
mp4: Add ISOBMFF to format description
2022-04-21 17:47:15 +02:00
Mattias Wadman
788b0ac197 rtmp,amf0: Improve decoders, aac asc, chunk stream interrupt, fix amf0 ecma arrays
rtmp: Handle chunk streams that have been interrupted (capture terminated in middle of stream etc)
rtmp: Decode AAC ASC
rtmp: Add ffmpeg client/server stream test
rtmp: Decode user control messages
rtmo: Decode all data messages
amf0: Fix ecma array decoding
2022-04-04 14:42:16 +02:00