Now ex "tovalue | .some.thing" on a decode value will make some.thing be jq value
instead of a decode value which woud be displayed as a decode treee, seems confusing.
I think this is more intuetive and make more sense.
Replaces []Format with a Group type.
A bit more type safe.
Breaking change for RegisterFormat, now takes a first argument that is a "single" format group.
Lots of naming cleanup.
This is also preparation for decode group argument which will enable doing intresting
probing, ex a format decoder could know it's decode as part of probe group (html could
be probed possibly), or have "arg probe" group for decoder who inspect args to know
if they should probe (-d /path/to/schema etc) to enable nice CLI-ergonomics.
Before it failed when passing a decode value, like an integer, string etc, as a object value
to an internal function (written in go), ex some_function({name: <decode value>}).
A future improvement could be to allow an internal function to say that it wants a "raw"
decode value etc and not one converted to primitive type.
This preserves the callstack on non-recoverable panics so that using
a debugger and fuzzing is much easier.
Add vscode debug config.
Remove fuzz stacktrace log workaround.
readline Config was used to pass completer function per readline call,
was changed in #612 and caused regression. Now use our own member in
stdOS to pass it instead.
Add a test but test script completer is implemented differently.
What it can do:
- Decodes records and most standard messages and extensions.
- Decryptes records and reassemples application data stream if a keylog is provided
and the cipher suite is supported.
- Supports most recommended and used ciphers and a bunch of older ones.
What it can't do:
- SSL v3 maybe supported, is similar to TLS 1.0, not tested.
- Decryption and renegotiation/cipher change.
- Record defragmentation not supported, seems rare over TCP.
- TLS 1.3
- SSL v2 but v2 compat header is supported.
- Some key exchange messages not decoded yet
Decryption code is heavly based on golang crypto/tls and zmap/zcrypto.
Will be base for decoding http2 and other TLS based on protocols.
Fixes#587
This will allow passing both cli options and format options to sub decoder.
Ex: pass keylog option to a tls decoder when decoding a pcap.
Ex: pass decode options to a format inside a http body inside a pcap.
Add ArgAs method to lookup argument based on type. This also makes the format
decode function have same signature as sub decoders in the decode API.
This change decode.Format a bit:
DecodeFn is now just func(d *D) any
DecodeInArg renamed to DefaultInArg
Feels less cluttered, easier to read and more consistent.
Still keep tovalue, tobytes etc that are more basic functions this
only renamed format related functions.
Also there is an exceptin for to/fromjson as it comes from jq.
Also fixes lots of spelling errors while reading thru.
Preparation to make decoder use less memory and API more type safe.
Now each scalar type has it's own struct type so it can store different
things and enables to have a scalar interface.
Also own types will enable experimenting with decode DLS designs like
using chained methods that are type aware.
Refactor mp4 decoder to be simpler and have fallback for unknown box type
Cleanup some old ilst hacks
Add generic string reader to decode API that takes an encoding parameters
Fix issue with JQValue:s in assig/update paths.
Rebase on upstream master, commits since last rebase:
c1a3ebd (tag: v0.12.10, origin/main) bump up version to 0.12.10
5294c9e update CHANGELOG.md for v0.12.10
319a797 update dependencies
20ca575 add notes on precedence of unary operators against variable binding
62ffdf7 fix break in try catch syntax
35c123b improve the order of test cases
fe0e20a improve performance of ascii_downcase, ascii_upcase functions
76648c7 improve match/2 to reduce syntax tree nodes just like splits/2
6a683bb improve the install targets in Makefile
6631dd3 escape the null character in csv, tsv, sh formats (close#200)
5253817 fix path value validation on getpath function (fix#198)
I think this is more intuitive but might in some case cause very large JSON output
but maybe that less common or expected. In does cases i think you either want to
use some other bits_format (md5, truncate, etc) or you delete/transform the jq value
before turn it into JSON.
Strings in gojq are binary safe so you can use to hold raw bytes. But note that
convert the binary into JSON is lossy, same as the JSON standard.
Add bits_format option documentation.
raw format was a hack to skip decoding to be able to get a binary using tobyte etc.
Now you can do fq -d bytes ... instead of fq -d raw 'tobytes | ...'
Make more sense to use a more general name when there is both info and xing
and possibly vbri in the future. Also both xing and info seems to have extension
fields
Remove some old tests that are not really needed anymore.
Rework time helpers to use new functions
Fix panic in tar decoder where sym value might be missing
Still not very happy about the API but it's getting better.
Rewrite cli query to use display as output query, same as for repl.
Without this "tovalue" values that don't print in the current eval
will end up in the paren eval and be written using an output without
a cancel context.
Before if you did toactual it behaved differently and even gave error for
a scalar that only had actual value set. Now both to{actal,sym} works
similar to tovalue but will force if actual or sym value should be used.
Markdown is used as is in online documentation and in cli the markdown decoder
is used to decode and the some jq code massages it into something cli friendly.
Was just too much of a mess to have doc in jq.
Will make it faster for struct with logs of fields and seems to
not cuase any significant difference for small structs.
All this really needs a rewrite somehow, maybe refactor into interfaces somehow? getting messy.
This removes bar column drawing responsility from already complicated dump code.
Start of dump code refactor that will enable configurable columns and proper column truncate/wrap.
Some mpeg versions use LSF (low sampling frequency) when encoding
low sample rates which changes a bit how the side info is layed out.
Also skip trying to separate data and other_data as after reading up
a bit on how main data begin and bit reservoir works i think it was done
incorrectly. To know the real audio data length i think you need to decode
the huffman tables. Instead just have a "audio_data" field which are the
bits with audo data for current and possibly following frames.
Update repl completion for new scope/0 function, now reports name/arity and is more correct
Support for experimental `rawstring` literals, similar to go raw string literals
Sticter number literals, now 0b,0o and 0x fail it they have invalud symbols, ex: 0b123
Later mappers should be able to rely on sym being a value of a certain type.
In this crash case tar.go had this:
d.FieldUTF8NullFixedLen("mtime", 12, scalar.SymUParseUint(8), scalar.DescriptionSymUUnixTime)
If mtime was empty string SymUParseUint was not even called so did not bail out running
all mappers.
Split fat macho into own decoder macho_fat. This also fixes issue with section
offset etc not being correct as they are from the start of each embedded file.
Make all address and offset field be in hex.
Decode __cstring, __ustring and __cfstring sections.
Fix LC_ENCRYPTION_INFO_64 missing pading issue.
Skip ranging for __bss and __common as they dont have any data in the file.
Simplifed magic handling a bit and add symbols.
Simplified state struct field, had redudant struct.
