As decoder now can know they are decoding as part of probing we can now
use some heuristics to see if we should decode as html.
The reason heuristics is needed is that x/html parser will alwaus succeed.
Add lazyre package to help delay compile of RE and make it concurrency safe.
What it can do:
- Decodes records and most standard messages and extensions.
- Decryptes records and reassemples application data stream if a keylog is provided
and the cipher suite is supported.
- Supports most recommended and used ciphers and a bunch of older ones.
What it can't do:
- SSL v3 maybe supported, is similar to TLS 1.0, not tested.
- Decryption and renegotiation/cipher change.
- Record defragmentation not supported, seems rare over TCP.
- TLS 1.3
- SSL v2 but v2 compat header is supported.
- Some key exchange messages not decoded yet
Decryption code is heavly based on golang crypto/tls and zmap/zcrypto.
Will be base for decoding http2 and other TLS based on protocols.
Fixes#587
raw format was a hack to skip decoding to be able to get a binary using tobyte etc.
Now you can do fq -d bytes ... instead of fq -d raw 'tobytes | ...'
Make more sense to use a more general name when there is both info and xing
and possibly vbri in the future. Also both xing and info seems to have extension
fields
Remove some old tests that are not really needed anymore.
Markdown is used as is in online documentation and in cli the markdown decoder
is used to decode and the some jq code massages it into something cli friendly.
Was just too much of a mess to have doc in jq.
Split fat macho into own decoder macho_fat. This also fixes issue with section
offset etc not being correct as they are from the start of each embedded file.
Make all address and offset field be in hex.
Decode __cstring, __ustring and __cfstring sections.
Fix LC_ENCRYPTION_INFO_64 missing pading issue.
Skip ranging for __bss and __common as they dont have any data in the file.
Simplifed magic handling a bit and add symbols.
Simplified state struct field, had redudant struct.
