1
1
mirror of https://github.com/wader/fq.git synced 2024-12-01 19:12:34 +03:00
Commit Graph

17 Commits

Author SHA1 Message Date
Mattias Wadman
c8f9cdc9d7 interp: Add string_truncate option
Was hardcoded to 50, now confiugurable also used affected by dd, dv etc
2024-04-08 17:45:10 +02:00
Mattias Wadman
051a70bd4b interp: Change bit ranges to use exclusive end
All other ranges and slicing uses exclusive end so i think it make sense
to make it consistent.

Update docs and add additional example for non-byte-aligned field.

Also fixes issue showing zero bit ranges as start-NA.
2023-10-20 15:37:26 +02:00
Mattias Wadman
1d14ea51cd matroska: Decode ebml date type
Also some refactor of shared date decode functions to allow unit
2023-10-17 11:10:48 +02:00
Platon Pronko
a3361e70ba bitcoin: fix witness item structs
This commit fixes witness item decoding such that they are represented
as structs with `item_size` and `item` fields, instead of a flat list
of raw bytes.
2023-05-15 18:36:26 +08:00
Mattias Wadman
a4a332bf4e formats: Clenaup naming a bit 2023-05-01 13:19:04 +02:00
Mattias Wadman
b08ef00dd1 decode,interp: Refactor format groups into a proper struct
Replaces []Format with a Group type.
A bit more type safe.
Breaking change for RegisterFormat, now takes a first argument that is a "single" format group.
Lots of naming cleanup.

This is also preparation for decode group argument which will enable doing intresting
probing, ex a format decoder could know it's decode as part of probe group  (html could
be probed possibly), or have "arg probe" group for decoder who inspect args to know
if they should probe (-d /path/to/schema etc) to enable nice CLI-ergonomics.
2023-04-29 20:02:34 +02:00
Mattias Wadman
980ecdba82 decode: Add float 80 reader
Rename *d.Bits to UintBits as it return a uint
Add *d.Bits that return []byte
2023-03-10 01:15:55 +01:00
Mattias Wadman
8e0dde03d0 decode: Support multiple format args and some rename and refactor
This will allow passing both cli options and format options to sub decoder.
Ex: pass keylog option to a tls decoder when decoding a pcap.
Ex: pass decode options to a format inside a http body inside a pcap.

Add ArgAs method to lookup argument based on type. This also makes the format
decode function have same signature as sub decoders in the decode API.

This change decode.Format a bit:
DecodeFn is now just func(d *D) any
DecodeInArg renamed to DefaultInArg
2023-02-18 21:38:51 +01:00
Mattias Wadman
a1bb630a2a doc,fq: Improve cli help and some cleanup 2023-02-15 11:04:39 +01:00
Mattias Wadman
9b81d4d3ab decode: More type safe API and split scalar into multiple types
Preparation to make decoder use less memory and API more type safe.
Now each scalar type has it's own struct type so it can store different
things and enables to have a scalar interface.
Also own types will enable experimenting with decode DLS designs like
using chained methods that are type aware.
2022-12-14 16:23:58 +01:00
Mattias Wadman
cb3dc80252 decode,tar: Add scalar description and Try* helpers
Rework time helpers to use new functions
Fix panic in tar decoder where sym value might be missing

Still not very happy about the API but it's getting better.
2022-10-04 17:18:51 +02:00
Mattias Wadman
a7a101ca67 doc,help: Nicer format help and move help tests into each format
Also add authors to avro and macho decoders

Generated with:
for i in $(go run . -r -n '_registry | [([([.files[][].name | split(".")[0]] | unique[]), (.formats[].name)] | count[] | select(.[1] == 2)[0]), (.formats[] | select(.decode_in_arg).name)] | unique[]'); do DIR=$(dirname $(find format -name $i.go)); echo "$ fq -h $i" > "$DIR/testdata/help_$i.fqtest" ; done
2022-09-22 23:07:58 +02:00
Mattias Wadman
004406de65 bitcoin_blkdat,bitcoin_block: Make sure there is a header if blkdat
Makes bitcoin_blkdat fails fast as it is part of probe group.
Speeds up reading a big JSON file etc.
2022-08-26 12:07:53 +02:00
Mattias Wadman
9a5fcc89f1 xml: Allow trailing <?procinstr?>
Also more context in error messages and refactor trim function
2022-08-11 23:48:29 +02:00
Mattias Wadman
29005c70bf interp,dump: Show address bar for root, nested roots and on format change
Also increase indent for nested roots a bit.

Makes it a bit easier to read i think.
2022-08-09 18:09:41 +02:00
Mattias Wadman
1ddea1ada3 interp,format: Refactor registry usage and use function helpers
Move registry to interp and add support for functions and filesystems.
This will be used later for allow formats to add own functions and fq code.

Add gojqextra function helpers to have more comfortable API to add functions.
Takes care of argument type casting and JQValue:s and some more things.

Refactor interp package to use new function helper and registry. Probably
fixes a bunch of JQValue bugs and other type errors.

Refactor out some mpeg nal things to mpeg format.

Refactor interp jq code into display.q and init.jq.

Remove undocumented aes_ctr funciton, was a test. Hopefully will add more crypto things laster.
2022-07-16 19:24:13 +02:00
Mattias Wadman
417255b72b bitcoin: Add blkdat, block, transcation and script decoder 2022-07-04 09:59:51 +02:00