David McDonald
708346784f
decode: remove dead code from ns_keyed_archiver
2022-12-17 01:52:46 -06:00
David McDonald
3ec0ba3f25
decode: add ns_keyed_archiver, restructure apple decoder into apple package
2022-12-17 01:47:27 -06:00
Mattias Wadman
1dea40e6af
mp4,doc: Add JSON box tree example and reorder a bit
2022-12-15 14:15:31 +01:00
Mattias Wadman
502e1e3059
Merge pull request #525 from wader/mp3-tags-split-decoder
...
mp3_frame_tags: Covert to decode group and split to mp3_frame_{xing,v…
2022-12-15 12:20:16 +01:00
Mattias Wadman
9e5a072e50
mp3_frame_tags: Covert to decode group and split to mp3_frame_{xing,vbri} decoders
2022-12-15 12:10:06 +01:00
Mattias Wadman
7b60b24ab7
mp4: Add pdin box support
2022-12-15 11:42:05 +01:00
Mattias Wadman
9b81d4d3ab
decode: More type safe API and split scalar into multiple types
...
Preparation to make decoder use less memory and API more type safe.
Now each scalar type has it's own struct type so it can store different
things and enables to have a scalar interface.
Also own types will enable experimenting with decode DLS designs like
using chained methods that are type aware.
2022-12-14 16:23:58 +01:00
Mattias Wadman
48522e3cb8
mp3_tags,mp3: Add VBRI header support and rename tags to tag as there is only one
2022-12-14 11:34:53 +01:00
Mattias Wadman
0c11ab5354
Merge pull request #521 from wader/mp4-ilst-namespace
...
mp4: Add namespace to mdta ilst boxes
2022-12-14 11:26:28 +01:00
Mattias Wadman
b1b3b63d27
mp4: Add namespace to mdta ilst boxes
2022-12-14 10:49:02 +01:00
David McDonald
7fb674b532
decode: unexport methods
2022-12-13 00:47:58 -06:00
David McDonald
f747873dbc
decode: implements lost and found for unreferenced objects
2022-12-12 23:48:33 -06:00
David McDonald
a873819ea2
decode: fixes endian of unicode strings
2022-12-10 21:04:56 -06:00
Mattias Wadman
0b9426976f
Merge pull request #493 from dgmcdona/macos_bookmark
...
Apple bookmarkData
2022-12-10 08:58:33 +01:00
David McDonald
bdb8166261
decode: removed unnecessary struct
2022-12-09 20:13:25 -06:00
Mattias Wadman
9ac453a17d
mp4: Fix typo in sample flags sample_is_depended_on description
2022-12-09 17:22:17 +01:00
Mattias Wadman
5fb81a148b
mp4: Add sym and description for traf sample flags
2022-12-09 16:22:43 +01:00
David McDonald
7e98b53894
decode: fix type on defer function call, test: add loop.fqtest
2022-12-08 15:01:16 -06:00
Mattias Wadman
1d6ce2c07a
mp4: Decode hint and hnti child boxes
2022-12-08 20:47:33 +01:00
Mattias Wadman
83ccedc506
mp4,decode: Properly decode ilst items (both mdta and mdir)
...
Refactor mp4 decoder to be simpler and have fallback for unknown box type
Cleanup some old ilst hacks
Add generic string reader to decode API that takes an encoding parameters
2022-12-08 15:56:22 +01:00
David McDonald
8d5dcff8e3
test: applebookmark: adds problematic test case
2022-12-08 01:48:59 -06:00
David McDonald
330d5f7f28
decode: apple_bookmark: simplifies flag decoding
2022-12-08 01:47:00 -06:00
David McDonald
a162e07b76
decode: minor change to method receiver name
2022-12-08 01:33:38 -06:00
David McDonald
04379df8d4
decode: revert decode.D back, place posLoopDetector in apple_bookmark
2022-12-07 16:25:34 -06:00
David McDonald
63a4e80c8b
test: fixed doc test
2022-12-06 19:46:25 -06:00
David McDonald
3942db79c2
pkg/decode/D: Adds PushAndPop, Push, Pop methods. doc: adds help_applebookmark.fqtestdecode: converts applebookmark to use new d.PushAndPop method
2022-12-06 19:40:28 -06:00
David McDonald
34c560c2f5
Merge branch 'master' of https://github.com/wader/fq into macos_bookmark
2022-12-05 22:01:44 -06:00
David McDonald
7ab446626e
decode: remove unused field from decoder, unused parens from torepr
2022-12-05 21:37:36 -06:00
Mattias Wadman
47a568e076
text,test: Unbeak base64 tests
2022-12-05 20:51:30 +01:00
Mattias Wadman
a23fe618b0
mp4: sidx version 1 segment_duration is s64
2022-12-05 17:04:05 +01:00
Mattias Wadman
2c4862c9fb
Merge pull request #498 from bitbears-dev/master
...
support tzif (time zone information format)
2022-12-04 13:21:19 +01:00
@0xb17bea125
fd22426b3d
tzif: add help_tzif.fqtest
2022-12-04 21:04:15 +09:00
@0xb17bea125
c4e7fc7919
tzif: moved document to tzif.md
2022-12-04 21:04:13 +09:00
@0xb17bea125
4481a77a1c
tzif: use scalar.Fn() to define a mapper ad hoc
2022-12-04 21:02:39 +09:00
@0xb17bea125
abde823a79
tzif: use PeekFindByte() to find end of the string
2022-12-04 21:02:39 +09:00
@0xb17bea125
2ee6360b97
support tzif (time zone information format)
2022-12-04 21:02:39 +09:00
Mattias Wadman
42debe5871
dev,doc,make: Cleanup makefile and have proper targets for *.md and *.svg
...
Update format dev docs to only generate README.md and doc/foramts.md to not
cause formats.svg conflicts.
2022-12-04 12:37:10 +01:00
David McDonald
b45f9fa67b
decode: improve stack push/pop
2022-12-03 15:27:59 -06:00
David McDonald
62f377c2fe
doc: fixes snippet for recursive bookmark searching
2022-12-03 15:24:49 -06:00
Mattias Wadman
6fc84a885c
doc,dev: Add more usage and dev tips
2022-12-03 19:00:47 +01:00
David McDonald
d199793af6
created stack type
2022-12-02 00:29:29 -06:00
David McDonald
b2aeac6ad3
updates bplist fq tests
2022-12-01 23:45:48 -06:00
David McDonald
55f4f1aa04
moved a flag bit fields into correct positions
2022-12-01 23:31:57 -06:00
David McDonald
c7d00b87b6
fixes unknown bit ranges
2022-12-01 23:31:57 -06:00
David McDonald
cb15b37175
added checks to prevent infinite looping and recursion
2022-12-01 23:31:57 -06:00
David McDonald
9bd65f93b7
migrates tests to per-sample files
2022-12-01 23:31:57 -06:00
David McDonald
7d13cf7312
adds flag parsing to applebookmark
2022-12-01 23:31:57 -06:00
David McDonald
1352598af9
removed commented out line
2022-12-01 23:31:57 -06:00
David McDonald
a23ac8f553
updates fqtest for torepr in apple_bookmarkdata
2022-12-01 23:31:57 -06:00
David McDonald
1e1fc551eb
fixed one more snake_case letter
2022-12-01 23:31:57 -06:00
David McDonald
d0b76cae23
fixes broken test and removes long link from markdown body
2022-12-01 23:31:57 -06:00
David McDonald
c244533540
added some sfl2 test files to bplist package
2022-12-01 23:31:57 -06:00
David McDonald
64351e8b44
fixes tests and adds torepr test
2022-12-01 23:31:57 -06:00
David McDonald
f7d7a49fb6
missed a letter on last commit - converting to snake_case
2022-12-01 23:31:57 -06:00
David McDonald
0c216dff48
refactors some decoder logic in apple_bookmark for better querying
2022-12-01 23:31:57 -06:00
David McDonald
44c91d8296
tweaks apple_bookmark markdown documentation
2022-12-01 23:31:57 -06:00
David McDonald
f535ad3de9
fixes spacing in jq files
2022-12-01 23:31:57 -06:00
David McDonald
a351c346f6
removes unused function
2022-12-01 23:31:57 -06:00
David McDonald
253033cc4c
fixes broken uid parsing in plist decoder
2022-12-01 23:31:57 -06:00
David McDonald
63a3ca20c4
removes underscore from apple_bookmark package name
2022-12-01 23:31:57 -06:00
David McDonald
4a28e44f44
changes decoder package name from bookmark to apple_bookmark
2022-12-01 23:31:57 -06:00
David McDonald
81269430fe
removed unnecessary conversions
2022-12-01 23:31:57 -06:00
David McDonald
d0b044c2f0
converts to snake_case and refactors decode helper
2022-12-01 23:31:57 -06:00
David McDonald
5146f28dbb
fixes broken test for all.fqtest
2022-12-01 23:31:57 -06:00
David McDonald
5b1455e777
removed unused function
2022-12-01 23:31:57 -06:00
David McDonald
6f4d1cb1de
updated documentation
2022-12-01 23:31:57 -06:00
David McDonald
dbc6fccdda
updated doc with apple reference
2022-12-01 23:31:57 -06:00
David McDonald
34db9d7fea
regenerated docs, added tests, fixed torepr
2022-12-01 23:31:57 -06:00
David McDonald
8f930aacee
forgot to add bookmark.jq in last commit
2022-12-01 23:31:57 -06:00
David McDonald
0287ffa4dc
decoding well but torepr needs work
2022-12-01 23:31:56 -06:00
David McDonald
71b17d0382
apple bookmarkdata decoder initial commit
2022-12-01 23:31:56 -06:00
Mattias Wadman
e77f776999
decode,interp: Rename unknown gap fields from "unknown#" to "gap#"
...
Think it makes it clearer and also less likely to collide with a field
name a deocder wants to use.
2022-12-01 20:43:30 +01:00
Mattias Wadman
8d69f1fb23
interp: Change default bits_format=string
...
I think this is more intuitive but might in some case cause very large JSON output
but maybe that less common or expected. In does cases i think you either want to
use some other bits_format (md5, truncate, etc) or you delete/transform the jq value
before turn it into JSON.
Strings in gojq are binary safe so you can use to hold raw bytes. But note that
convert the binary into JSON is lossy, same as the JSON standard.
Add bits_format option documentation.
2022-12-01 17:49:34 +01:00
Mattias Wadman
42d9f2c27d
interp,help: Properly count line length when breaking on whole words
...
Whitespace between words were not counted
2022-11-29 18:59:32 +01:00
Mattias Wadman
ef2d5232ba
mp4: Add png mapping
...
Also support samples decode for tracks without hdlr box
2022-11-25 19:49:18 +01:00
Mattias Wadman
d75748d816
mp4: Decode more sample flags
2022-11-23 14:29:13 +01:00
Mattias Wadman
646f32d563
matroska: Fix path tests and make _tree_path more robust
2022-11-20 20:16:41 +01:00
Mattias Wadman
c93301fc70
raw,bits,bytes: Replace raw format with bits and bytes format that decode to a binary
...
raw format was a hack to skip decoding to be able to get a binary using tobyte etc.
Now you can do fq -d bytes ... instead of fq -d raw 'tobytes | ...'
2022-11-20 19:51:12 +01:00
Mattias Wadman
0cf46e1196
wav,avi,avc_au: Add avi decoder and refactor wav decoder
...
avi and wav now share common riff decode code
avc_au decoder can now handle annexb format
2022-11-19 13:11:27 +01:00
Mattias Wadman
c3a0686c9e
mp3_frame_tags: Refactor and rename xing format to mp3_frame_tags
...
Make more sense to use a more general name when there is both info and xing
and possibly vbri in the future. Also both xing and info seems to have extension
fields
Remove some old tests that are not really needed anymore.
2022-11-16 17:32:03 +01:00
Mattias Wadman
2de875393a
mp3_frame: Fix issue calc frame size for some configs
...
Seems have to use the 144000*bitrate/sameplrate to get correct size or
one would miss one byte (odd size rounded to even?)
2022-11-16 15:55:10 +01:00
Mattias Wadman
f66e224452
id3v2: In the wild COMM and TXXX frame might not have a null terminator
2022-10-20 22:07:37 +02:00
Mattias Wadman
fb2a2b949c
flac,fuzz: Fatal error on negative partition sample count
2022-10-18 13:40:48 +02:00
Mattias Wadman
2606916753
yaml,fuzz: gojq.Normalize value to fix type panic
2022-10-11 19:22:49 +02:00
Mattias Wadman
7859be1e5d
flac_frame: Properly decode zero escape sample size
2022-10-11 17:05:19 +02:00
Mattias Wadman
3a578152f2
Merge pull request #459 from wader/decode-s-must-least-1bit
...
decode,fuzz: Signed integer (S) read require at least one bit
2022-10-11 12:41:28 +02:00
Mattias Wadman
bafd1f564e
decode,fuzz: Signed integer (S) read require at least one bit
...
Otherwise there is no sign bit and it will panic on shifting.
Maybe should require 2? now -0 and +1 hmm
2022-10-11 12:12:28 +02:00
Mattias Wadman
cef4245b49
fuzz: make fuzz GROUP=mp4 to fuzz one group
2022-10-10 21:07:10 +02:00
Mattias Wadman
e9aa4837e9
Merge pull request #454 from dgmcdona/reference_size_bug_fix
...
fixes reference calculation to use reference size from trailer
2022-10-09 21:48:12 +02:00
David McDonald
b09d61162b
makes dictionary key type checking more sensible
2022-10-09 13:34:00 -05:00
Mattias Wadman
2a86d323b4
doc,rtmp,pcap,markdown: Add more examples
2022-10-09 19:18:14 +02:00
Mattias Wadman
e748079e87
matroska: Update spec and regenerate
...
The new type "TargetType" used in labels is bit confusing, should be handled differently?
2022-10-08 15:45:46 +02:00
David McDonald
ac86f9318b
fixes signed integer parsing
2022-10-07 21:33:26 -05:00
David McDonald
2b3adbe8d6
renames test data file
2022-10-07 18:08:20 -05:00
David McDonald
4ee7dd8a38
changes Errorf to Fatalf on infinite loops
2022-10-07 18:04:07 -05:00
David McDonald
4d8dd5c5e3
adds check for recursion in decodeReference, adds test to verify fix
2022-10-07 15:07:05 -05:00
David McDonald
413d425079
gofmt
2022-10-07 13:16:17 -05:00
David McDonald
fb1a91ac3c
drop indented else block per lint
2022-10-07 13:15:22 -05:00
David McDonald
236fbc17b5
fixes reference calculation to use reference size from trailer
2022-10-07 13:07:51 -05:00
David McDonald
4dd594c14e
fixes bad path in test output
2022-10-07 00:50:48 -05:00
