1
1
mirror of https://github.com/wader/fq.git synced 2024-12-28 08:02:28 +03:00
Commit Graph

23 Commits

Author SHA1 Message Date
Mattias Wadman
c51271399c interp: Add skip_gaps option for tovalue/-V
Skips gap fields in struct and arrays.

Gaps fields are bit ranges that a decoder did not add any fields for.
Note that skipping gaps in arrays will affect indexes.
2023-04-30 13:04:39 +02:00
Mattias Wadman
9b81d4d3ab decode: More type safe API and split scalar into multiple types
Preparation to make decoder use less memory and API more type safe.
Now each scalar type has it's own struct type so it can store different
things and enables to have a scalar interface.
Also own types will enable experimenting with decode DLS designs like
using chained methods that are type aware.
2022-12-14 16:23:58 +01:00
Mattias Wadman
e77f776999 decode,interp: Rename unknown gap fields from "unknown#" to "gap#"
Think it makes it clearer and also less likely to collide with a field
name a deocder wants to use.
2022-12-01 20:43:30 +01:00
Mattias Wadman
cb3dc80252 decode,tar: Add scalar description and Try* helpers
Rework time helpers to use new functions
Fix panic in tar decoder where sym value might be missing

Still not very happy about the API but it's getting better.
2022-10-04 17:18:51 +02:00
David McDonald
a77cec921f Added documentation and tests, fixed bad date parsing 2022-09-23 20:23:23 -05:00
David McDonald
5711f29084 Code fixes from PR, still need to add tests and testdata 2022-09-23 20:21:32 -05:00
Mattias Wadman
0d44b9376c tar: Some number fields can be empty 2022-08-15 14:57:16 +02:00
Mattias Wadman
0cd846a18c *extra: Rename <pkg>extra to just <pkg>ex and refactor to use generics 2022-08-12 16:29:47 +02:00
Mattias Wadman
502f451cc3 decode: Refactor to use scalar type assert helper 2022-08-08 18:09:49 +02:00
Mattias Wadman
5d9ffead52 decode,scalar: Map empty string also else sym might ends up nil
Later mappers should be able to rely on sym being a value of a certain type.
In this crash case tar.go  had this:
d.FieldUTF8NullFixedLen("mtime", 12, scalar.SymUParseUint(8), scalar.DescriptionSymUUnixTime)
If mtime was empty string SymUParseUint was not even called so did not bail out running
all mappers.
2022-08-04 12:23:31 +02:00
Mattias Wadman
417255b72b bitcoin: Add blkdat, block, transcation and script decoder 2022-07-04 09:59:51 +02:00
Mattias Wadman
7b27e506f1 mp4,bitio: Fix broken pssh decoding and add proper reader cloning to bitio
PSSH decoding probably got broken in some refactoring. system id is 16 bytes,
also scalar mapper should use cloned reader to not affect reader which in the case
broke reading system id bytes.

Add test with playread and widevine pssh boxes

Related to #282
2022-05-28 13:42:38 +02:00
Mattias Wadman
e9d9f8aef9 fq: Use go 1.18
Rename s/interface{}/any/g
Preparation for using generics in decode API and native jq funcations etc
Remove some unused linter ignores as linter has been fixed
2022-05-20 15:23:16 +02:00
Mattias Wadman
76161a1b99 scalar,mp4,gzip,tar: Add timestamp to description
Scalar api for this needs some work
2022-05-07 13:15:13 +02:00
Mattias Wadman
9133f0e527 scalar: Add *Fn type to map value and clearer naming
Should replace most of this with generics at some point
2022-05-07 12:46:34 +02:00
Mattias Wadman
f8fd6b7f48 mp4: Add more HEIF boxes
Add scalar.*ToDesceription mapper and refactor to use it
2022-04-07 19:11:49 +02:00
Mattias Wadman
b86da7aecc doc: Add inital decoder API documentation 2022-02-12 16:13:29 +01:00
Mattias Wadman
7c5215347d bitio,decode: Refactor bitio usage and make buffer slicing more correct
Remove bitio.Buffer layer. bitio.Buffer was a kitchen sink layer with helpers
now it's just a buffer and most functions have been moved to decode instead.

bitio package now only have primitive types and functions simialar to standard
library io and bytes packages.

Make nearly eveything internally use bitio.Bit* interfaces so that slicing work
correctly this will also make it possible to start experimenting with more
complicated silcing helpers, ex things like:
breplace(.header.bitrate; 123) to get a new buffer with bitrate changed.
2022-02-04 21:41:53 +01:00
Mattias Wadman
6cd1c38fbf decode,scalar: Add scalar.Str{Uint/Int/F}ToSym to parse numbers 2022-01-28 17:18:50 +01:00
Mattias Wadman
1383b411ae decode,interp: Add arbitrary large integer support (BigInt)
Was already handled in fq in various places as gojq uses them

Update msgpack to support negative integers that can't represented as int64
Rename read try* number functions to make them more explicit
2022-01-15 19:00:42 +01:00
Mattias Wadman
681dbc26b7 elf,ar: Add ar decoder, improved elf decoder
Add x86, 386, arm64, arm v6, arm v7 linux elf test files
2022-01-05 22:05:09 +01:00
Mattias Wadman
c2131bb4a0 flac: Cleanup scalar usage and fix incorrect sample rates
Also split DisplayFormat into actual and sym
2021-12-08 00:39:52 +01:00
Mattias Wadman
2fc0a71a47 decode: Refactor scalar usage
Move scalar into own package.
Split scalar code into decode related scalar code (that reads etc) and
scalar code that just transform the scalar value.
Use a scalar.Mapper interface instead of just a function.
Make mappers, assert and validat impement the interface.
2021-12-02 17:39:26 +01:00