raw format was a hack to skip decoding to be able to get a binary using tobyte etc.
Now you can do fq -d bytes ... instead of fq -d raw 'tobytes | ...'
Will make it faster for struct with logs of fields and seems to
not cuase any significant difference for small structs.
All this really needs a rewrite somehow, maybe refactor into interfaces somehow? getting messy.
Split fat macho into own decoder macho_fat. This also fixes issue with section
offset etc not being correct as they are from the start of each embedded file.
Make all address and offset field be in hex.
Decode __cstring, __ustring and __cfstring sections.
Fix LC_ENCRYPTION_INFO_64 missing pading issue.
Skip ranging for __bss and __common as they dont have any data in the file.
Simplifed magic handling a bit and add symbols.
Simplified state struct field, had redudant struct.
Doing it thru a propery in the decode fn feels a bit hidden and will
also not get set on failed decoding.
Now array is not range sorted, logic is you care about index number and ordering.
Struct is range sorted as you will prefer to fields by name.
json, yaml, toml, xml, html, csv are now normal formats and most of them also particiate
in probing (not html and csv).
Also fixes a bunch of bugs in to/fromxml, to/fromjq etc.
Move registry to interp and add support for functions and filesystems.
This will be used later for allow formats to add own functions and fq code.
Add gojqextra function helpers to have more comfortable API to add functions.
Takes care of argument type casting and JQValue:s and some more things.
Refactor interp package to use new function helper and registry. Probably
fixes a bunch of JQValue bugs and other type errors.
Refactor out some mpeg nal things to mpeg format.
Refactor interp jq code into display.q and init.jq.
Remove undocumented aes_ctr funciton, was a test. Hopefully will add more crypto things laster.
Decode API design is that Try* returns error non-Try panics.
Also rename Must* as they should panic and introduce some new Try* functions
that were missing.
PSSH decoding probably got broken in some refactoring. system id is 16 bytes,
also scalar mapper should use cloned reader to not affect reader which in the case
broke reading system id bytes.
Add test with playread and widevine pssh boxes
Related to #282
Rename s/interface{}/any/g
Preparation for using generics in decode API and native jq funcations etc
Remove some unused linter ignores as linter has been fixed
Some formats might want to control child order
mp4: Keep tracks in track id order
dns: Keep label component order
elf: Keep seciton order
macho: Keep command and section order
interp: Refactor format help and also include options
interp: Add -o name=@path to load file content as value (not documented yet, might change)
interp,decode: Expose decode out value as _out (might change)
interp: Refactor foramts.jq into format_{decode,func,include}.jq
interp: Refactor torepr into _format_func for generic format function overloading
interp: Refactor -o options parsing to be more generic and collect unknowns options to be used as format options
decode of decode alises
func for format overloaded functions
include for format specific jq functions (also _help, torepr etc)
flac_frame: Add bits_per_sample option
mp3: Add max_unique_header_config and max_sync_seek options
mp4: Add decode_samples and allow_truncate options
avc_au: Has length_size option
hevc_au: Has length_size option
aac_frame: Has object_typee option
doc: Rewrite format doc generation, less hack more jq
Rename buffer to binary. Still some work left what to call buffer/binary in decode code.
Document decode value and binary type
Fix proper unit padding for tobytes and add still undocumenated extra padding argument.
Add some additional binary tests
Remove bitio.Buffer layer. bitio.Buffer was a kitchen sink layer with helpers
now it's just a buffer and most functions have been moved to decode instead.
bitio package now only have primitive types and functions simialar to standard
library io and bytes packages.
Make nearly eveything internally use bitio.Bit* interfaces so that slicing work
correctly this will also make it possible to start experimenting with more
complicated silcing helpers, ex things like:
breplace(.header.bitrate; 123) to get a new buffer with bitrate changed.
Was already handled in fq in various places as gojq uses them
Update msgpack to support negative integers that can't represented as int64
Rename read try* number functions to make them more explicit
