This will allow passing both cli options and format options to sub decoder.
Ex: pass keylog option to a tls decoder when decoding a pcap.
Ex: pass decode options to a format inside a http body inside a pcap.
Add ArgAs method to lookup argument based on type. This also makes the format
decode function have same signature as sub decoders in the decode API.
This change decode.Format a bit:
DecodeFn is now just func(d *D) any
DecodeInArg renamed to DefaultInArg
Preparation to make decoder use less memory and API more type safe.
Now each scalar type has it's own struct type so it can store different
things and enables to have a scalar interface.
Also own types will enable experimenting with decode DLS designs like
using chained methods that are type aware.
raw format was a hack to skip decoding to be able to get a binary using tobyte etc.
Now you can do fq -d bytes ... instead of fq -d raw 'tobytes | ...'
Before if you did toactual it behaved differently and even gave error for
a scalar that only had actual value set. Now both to{actal,sym} works
similar to tovalue but will force if actual or sym value should be used.
Will make it faster for struct with logs of fields and seems to
not cuase any significant difference for small structs.
All this really needs a rewrite somehow, maybe refactor into interfaces somehow? getting messy.
Feels like what one would expect when doing to_entries on a decode value struct etc,
that is get the pairs in same order as decoder wants.
Quite sure this used to the case but was change to sorted key order when
to_entries was move from bein implemented in jq to be internal for performance
reasons. Now internal gojq to_entries has a special case for JQValue objects.
Also use new gojq.JQType* constants for type strings and switch binary type
to look more like strings. Can still use _exttype to get "binary".
json, yaml, toml, xml, html, csv are now normal formats and most of them also particiate
in probing (not html and csv).
Also fixes a bunch of bugs in to/fromxml, to/fromjq etc.
Move registry to interp and add support for functions and filesystems.
This will be used later for allow formats to add own functions and fq code.
Add gojqextra function helpers to have more comfortable API to add functions.
Takes care of argument type casting and JQValue:s and some more things.
Refactor interp package to use new function helper and registry. Probably
fixes a bunch of JQValue bugs and other type errors.
Refactor out some mpeg nal things to mpeg format.
Refactor interp jq code into display.q and init.jq.
Remove undocumented aes_ctr funciton, was a test. Hopefully will add more crypto things laster.
PSSH decoding probably got broken in some refactoring. system id is 16 bytes,
also scalar mapper should use cloned reader to not affect reader which in the case
broke reading system id bytes.
Add test with playread and widevine pssh boxes
Related to #282
Rename s/interface{}/any/g
Preparation for using generics in decode API and native jq funcations etc
Remove some unused linter ignores as linter has been fixed
interp: Refactor format help and also include options
interp: Add -o name=@path to load file content as value (not documented yet, might change)
interp,decode: Expose decode out value as _out (might change)
interp: Refactor foramts.jq into format_{decode,func,include}.jq
interp: Refactor torepr into _format_func for generic format function overloading
interp: Refactor -o options parsing to be more generic and collect unknowns options to be used as format options
decode of decode alises
func for format overloaded functions
include for format specific jq functions (also _help, torepr etc)
flac_frame: Add bits_per_sample option
mp3: Add max_unique_header_config and max_sync_seek options
mp4: Add decode_samples and allow_truncate options
avc_au: Has length_size option
hevc_au: Has length_size option
aac_frame: Has object_typee option
doc: Rewrite format doc generation, less hack more jq
Upstream changes:
Many performance improvements
Error message improvments
Lots of refactoring
Most of the JQValue interface changes in gojq fork had to be reworked but
resultet in a much nicer and cleaner changeset.
fq changes:
Assignment to JQValue (like a decode value) now shallowly converts the value into
a jq value before assigning. Was a bit hacky as it was and this makes JQValue
behave more like real jq values. This also fixes some advanced indexing issues.
Actual custom path updates will be something for the future.
Speeds up interp quite a bit.
Also good as i've start to use more includes to make code easier to follow
where thigns come from and also makes jq-lsp happier.
Also rename evalContext to evalInstace to make it less confused with context.
Rename buffer to binary. Still some work left what to call buffer/binary in decode code.
Document decode value and binary type
Fix proper unit padding for tobytes and add still undocumenated extra padding argument.
Add some additional binary tests
Remove bitio.Buffer layer. bitio.Buffer was a kitchen sink layer with helpers
now it's just a buffer and most functions have been moved to decode instead.
bitio package now only have primitive types and functions simialar to standard
library io and bytes packages.
Make nearly eveything internally use bitio.Bit* interfaces so that slicing work
correctly this will also make it possible to start experimenting with more
complicated silcing helpers, ex things like:
breplace(.header.bitrate; 123) to get a new buffer with bitrate changed.
Was already handled in fq in various places as gojq uses them
Update msgpack to support negative integers that can't represented as int64
Rename read try* number functions to make them more explicit
