1
1
mirror of https://github.com/wader/fq.git synced 2024-11-26 21:55:57 +03:00
Commit Graph

162 Commits

Author SHA1 Message Date
Mattias Wadman
23ae4d978b decode,interp: Make synthetic values more visible and not act as decode values 2023-10-07 17:40:16 +02:00
Mattias Wadman
bedd719b3a mp4: Decode emsg box 2023-08-31 17:33:18 +02:00
Mattias Wadman
312d8078a7 mp4: Decode cslg box 2023-08-31 15:00:29 +02:00
Mattias Wadman
e50028ac00 mp4,mpeg_es: Decode iods box and MP4_IOD_Tag OD 2023-08-30 14:57:41 +02:00
Mattias Wadman
0d0147643a gomod: Update x/exp and gomarkdown
Add cmp package from go 1.21 to have cmp.Compare to make sort easier
2023-08-09 14:36:51 +02:00
Mattias Wadman
97194ad8bc mp4: Nicer major brand and handle some qt brand short strings better 2023-07-26 12:15:51 +02:00
Sergei Kuzmin
e869d8af03 sgpd box entries parsing
Previously entries were parsed as raw bytes, now KIDs/IVs are
explicitely parsed.
2023-06-20 21:57:27 -07:00
Mattias Wadman
1da8069190 mp4: udta: Improve length/lang box probe and support empty value 2023-05-17 11:55:13 +02:00
Mattias Wadman
493848a7e5 mp4: Use correct epoch for quicktime timestamps
Somehow was 1904-01-04 should be 1904-01-01
2023-05-16 13:10:56 +02:00
Mattias Wadman
2c505feec8 help,markdown: Rewrote and made text rendering nicer
Plan is to use it for man page etc and also some ansi renderer
2023-05-08 00:07:56 +02:00
Mattias Wadman
a4a332bf4e formats: Clenaup naming a bit 2023-05-01 13:19:04 +02:00
Mattias Wadman
7adc1e7030 mp4: Better description for QuickTime terminator atom
Addition improvement could be to only allow it in quicktime ftyp brand
and also not as a root book. Currently trailing zero bytes could be
mistaken as a terminator atom.
2023-05-01 12:32:25 +02:00
Mattias Wadman
b08ef00dd1 decode,interp: Refactor format groups into a proper struct
Replaces []Format with a Group type.
A bit more type safe.
Breaking change for RegisterFormat, now takes a first argument that is a "single" format group.
Lots of naming cleanup.

This is also preparation for decode group argument which will enable doing intresting
probing, ex a format decoder could know it's decode as part of probe group  (html could
be probed possibly), or have "arg probe" group for decoder who inspect args to know
if they should probe (-d /path/to/schema etc) to enable nice CLI-ergonomics.
2023-04-29 20:02:34 +02:00
Mattias Wadman
3c6d31b0fe mp4: ctts,infe,iinf,trun: More ISOMFF version handling 2023-04-17 11:43:54 +02:00
Mattias Wadman
980ecdba82 decode: Add float 80 reader
Rename *d.Bits to UintBits as it return a uint
Add *d.Bits that return []byte
2023-03-10 01:15:55 +01:00
Mattias Wadman
cc52a4419d id3v2: Decode subframes for CTOC and add struct for headers 2023-03-05 19:29:22 +01:00
Mattias Wadman
dec433fc53 help,markdown: Fix double line breaks when converting to text
Also fix ugly last line break hack
2023-03-02 23:38:11 +01:00
Mattias Wadman
8e0dde03d0 decode: Support multiple format args and some rename and refactor
This will allow passing both cli options and format options to sub decoder.
Ex: pass keylog option to a tls decoder when decoding a pcap.
Ex: pass decode options to a format inside a http body inside a pcap.

Add ArgAs method to lookup argument based on type. This also makes the format
decode function have same signature as sub decoders in the decode API.

This change decode.Format a bit:
DecodeFn is now just func(d *D) any
DecodeInArg renamed to DefaultInArg
2023-02-18 21:38:51 +01:00
Mattias Wadman
a1bb630a2a doc,fq: Improve cli help and some cleanup 2023-02-15 11:04:39 +01:00
Mattias Wadman
b60aceca9e matroska: Add decode_samples option
Also change the option help a bit, maybe medid and support is confusing, we
might evetually decode non-audio/video things.
2023-02-08 11:31:41 +01:00
Mattias Wadman
a2cdb3d6c9
Merge pull request #558 from wader/mp4-udta-with-no-length
mp4: udta: Handle box with value rest of box
2023-01-25 16:34:58 +01:00
Mattias Wadman
6340365841 mp4: udta: Handle box with value rest of box
Try distinguish by probing length field. Should probably be improved, what does ffmpeg do?

Regression from c3e3b3e90d #553
2023-01-25 16:24:28 +01:00
Mattias Wadman
b04a650bae flac_picture,mpeg: Fix trailing ")" typo in map sym and description 2023-01-25 14:01:34 +01:00
Mattias Wadman
d645e71008
Merge pull request #554 from wader/mp4-ftyp-qt-minor-desc
mp4: Decode qt minor verison as YYYY.MM description
2023-01-16 12:23:42 +01:00
Mattias Wadman
f386a5158e mp4: Decode qt minor verison as YYYY.MM description 2023-01-16 12:13:19 +01:00
Mattias Wadman
c3e3b3e90d mp4: Decode udta metadata boxes without meta box 2023-01-16 12:03:53 +01:00
Mattias Wadman
c2795a7c2f
Merge pull request #552 from wader/mp4-tkhd-flags
mp4: Decode tkhd flags
2023-01-13 18:02:53 +01:00
Mattias Wadman
3555dc6721 mp4: Decode tkhd flags 2023-01-13 17:49:03 +01:00
Mattias Wadman
c49012dbca mp4: sgpd,sbgp: Change grouping_type to a string
Is defined as int32 but is a string in practive it seems
2023-01-05 19:18:48 +01:00
Mattias Wadman
e3ae1440c9 interp: Rename to/from<format> functions to to_/from_<format>
Feels less cluttered, easier to read and more consistent.

Still keep tovalue, tobytes etc that are more basic functions this
only renamed format related functions.
Also there is an exceptin for to/fromjson as it comes from jq.

Also fixes lots of spelling errors while reading thru.
2022-12-21 17:48:39 +01:00
Mattias Wadman
a85da29544 decode: Make FieldFormat usage more consistent
Will make it easier to refactor later on
2022-12-19 19:12:57 +01:00
Mattias Wadman
1dea40e6af mp4,doc: Add JSON box tree example and reorder a bit 2022-12-15 14:15:31 +01:00
Mattias Wadman
7b60b24ab7 mp4: Add pdin box support 2022-12-15 11:42:05 +01:00
Mattias Wadman
9b81d4d3ab decode: More type safe API and split scalar into multiple types
Preparation to make decoder use less memory and API more type safe.
Now each scalar type has it's own struct type so it can store different
things and enables to have a scalar interface.
Also own types will enable experimenting with decode DLS designs like
using chained methods that are type aware.
2022-12-14 16:23:58 +01:00
Mattias Wadman
b1b3b63d27 mp4: Add namespace to mdta ilst boxes 2022-12-14 10:49:02 +01:00
Mattias Wadman
9ac453a17d mp4: Fix typo in sample flags sample_is_depended_on description 2022-12-09 17:22:17 +01:00
Mattias Wadman
5fb81a148b mp4: Add sym and description for traf sample flags 2022-12-09 16:22:43 +01:00
Mattias Wadman
1d6ce2c07a mp4: Decode hint and hnti child boxes 2022-12-08 20:47:33 +01:00
Mattias Wadman
83ccedc506 mp4,decode: Properly decode ilst items (both mdta and mdir)
Refactor mp4 decoder to be simpler and have fallback for unknown box type
Cleanup some old ilst hacks
Add generic string reader to decode API that takes an encoding parameters
2022-12-08 15:56:22 +01:00
Mattias Wadman
a23fe618b0 mp4: sidx version 1 segment_duration is s64 2022-12-05 17:04:05 +01:00
Mattias Wadman
e77f776999 decode,interp: Rename unknown gap fields from "unknown#" to "gap#"
Think it makes it clearer and also less likely to collide with a field
name a deocder wants to use.
2022-12-01 20:43:30 +01:00
Mattias Wadman
ef2d5232ba mp4: Add png mapping
Also support samples decode for tracks without hdlr box
2022-11-25 19:49:18 +01:00
Mattias Wadman
d75748d816 mp4: Decode more sample flags 2022-11-23 14:29:13 +01:00
Mattias Wadman
c93301fc70 raw,bits,bytes: Replace raw format with bits and bytes format that decode to a binary
raw format was a hack to skip decoding to be able to get a binary using tobyte etc.
Now you can do fq -d bytes ... instead of fq -d raw 'tobytes | ...'
2022-11-20 19:51:12 +01:00
Mattias Wadman
cb3dc80252 decode,tar: Add scalar description and Try* helpers
Rework time helpers to use new functions
Fix panic in tar decoder where sym value might be missing

Still not very happy about the API but it's getting better.
2022-10-04 17:18:51 +02:00
Mattias Wadman
20a15372ba mp4: Fix data_format typo 2022-10-02 23:31:38 +02:00
Mattias Wadman
a7a101ca67 doc,help: Nicer format help and move help tests into each format
Also add authors to avro and macho decoders

Generated with:
for i in $(go run . -r -n '_registry | [([([.files[][].name | split(".")[0]] | unique[]), (.formats[].name)] | count[] | select(.[1] == 2)[0]), (.formats[] | select(.decode_in_arg).name)] | unique[]'); do DIR=$(dirname $(find format -name $i.go)); echo "$ fq -h $i" > "$DIR/testdata/help_$i.fqtest" ; done
2022-09-22 23:07:58 +02:00
Mattias Wadman
2655ba09c5 mp4: More codec names (from mp4ra.org) 2022-09-12 11:25:43 +02:00
Mattias Wadman
abd19ed89b doc: Fix format sections a bit 2022-09-11 09:55:56 +02:00
Mattias Wadman
87b2c6c10c help,doc: Use markdown for format documentation again
Markdown is used as is in online documentation and in cli the markdown decoder
is used to decode and the some jq code massages it into something cli friendly.

Was just too much of a mess to have doc in jq.
2022-09-11 00:53:29 +02:00