1
1
mirror of https://github.com/wader/fq.git synced 2024-11-29 12:14:17 +03:00
Commit Graph

185 Commits

Author SHA1 Message Date
Mattias Wadman
8e47fb1a85 doc,matroska: Fix filesname in example 2022-01-29 12:20:47 +01:00
Mattias Wadman
a8664ed5ba doc: Add per format documentation 2022-01-29 12:01:36 +01:00
Mattias Wadman
6cd1c38fbf decode,scalar: Add scalar.Str{Uint/Int/F}ToSym to parse numbers 2022-01-28 17:18:50 +01:00
Mattias Wadman
8d2d88f4d0 mp3: Decrease max sync seek length between frames to 4k
This might break mp3s with large id3v2 chunks embedded into the stream but
i think that is rare and probably means it is a dump of a mp3 radio stream etc.

An alternative approach is to also look for id3v2 etc between frames but then the mp3
root value probably should be an array of mix of things instead of a struct, or rename
header/footer to metadata?

Related to #103
2022-01-25 16:08:43 +01:00
Mattias Wadman
4f490e5618
Merge pull request #99 from wader/mpeg-spu-fatal-infinit-loop
mpeg_spu: Fatal error on infinite loop
2022-01-24 17:53:56 +01:00
Mattias Wadman
b8efd8e549 fuzz: Fuzz all formats
Fix string readers to not panic on make slice with bogus sizes
2022-01-24 16:56:24 +01:00
Mattias Wadman
febce5a5a4 mpeg_spu: Fatal error on infinite loop 2022-01-24 16:53:55 +01:00
Mattias Wadman
d555c3248c mp4,fuzz: Fatal error on infinite sgpd box entries
Also rename array to entries
2022-01-23 14:24:57 +01:00
Mattias Wadman
d1943dad49 pcapng,fuzz: Fix infinite loop by fatal error on block length <= 0
Also add dev/fuzzbytes.go tool to convert fuzz input to raw bytes
2022-01-23 13:34:56 +01:00
Mattias Wadman
778a1a4178 zip: Assert signature not validate 2022-01-22 16:26:03 +01:00
Mattias Wadman
85371173b5 id3v2: Should assert not validate magic
Fixes issue with mp3 frame that happens to have a 2,3 or 4 at byte 3 being seen as id3v2 header
2022-01-22 16:16:29 +01:00
Mattias Wadman
cf8a50c150 decode: Use stable sort for values to not change order or values with same range start
Some decoders might relay on values added with same start to be kept in same order.
2022-01-20 16:21:50 +01:00
Mattias Wadman
ba1edef793 tar: Allow more than 2 zero end blocks at end 2022-01-18 12:54:50 +01:00
Mattias Wadman
39d4d846d7
Merge pull request #86 from wader/tar-dont-assume-endmarker
tar: Don't assume there is a end marker
2022-01-18 12:20:59 +01:00
Mattias Wadman
5921d76bf0 tar: Don't assume there is a end marker
GNU tar docs says:
At the end of the archive file there are two 512-byte blocks filled with binary zeros as an
end-of-file marker. A reasonable system should write such end-of-file marker at the end of
an archive, but must not assume that such a block exists when reading an archive.
In particular GNU tar always issues a warning if it does not encounter it.

Also clean up API usage a bit
2022-01-18 12:12:34 +01:00
Mattias Wadman
263f1aeb66 flac: Don't allow zero subframe sample size
Spec says sample size can be 4-32 but that is probably before correcting for waste bits
2022-01-18 11:58:57 +01:00
Mattias Wadman
61bf2ce644 mp4: Refactor sample decode into something more sane
Also fatal error instead of silent ignore if index goes outside stsc or stco tables.
2022-01-17 13:38:11 +01:00
Mattias Wadman
548a065388 decode,interp: Finish up nil value support
Update msgpack, bson and cbor to use it.
2022-01-17 10:49:46 +01:00
Mattias Wadman
edad481878 num,mathextra: Rename num package to mathextra
Think it makes more sense
2022-01-15 19:00:42 +01:00
Mattias Wadman
1383b411ae decode,interp: Add arbitrary large integer support (BigInt)
Was already handled in fq in various places as gojq uses them

Update msgpack to support negative integers that can't represented as int64
Rename read try* number functions to make them more explicit
2022-01-15 19:00:42 +01:00
Mattias Wadman
4a1e8590fa mp4: Improved stsz handling
Track size/count instead of just sizes which should decrease memory usage
making count sanity check unnecessary.

Fixes issue with huge mp4 files (80gb+) with lots of samples.
2022-01-15 17:51:10 +01:00
Mattias Wadman
729a6cae1e formats: Sort and make lists less likely to cause collision 2022-01-12 18:35:28 +01:00
Mattias Wadman
0b0f28e966 cbor: Add decoder
Does not decode sematic tag types
Also fixes broken float16 support

Fixes #71
2022-01-12 17:56:03 +01:00
Mattias Wadman
149cb3f45a interp: Add torepr/0 that converts decode value into what it reptresents
Ex: fq -d msgpack torepr file.msgpack
Willoutput the JSON representation of the msgpack

Make per format *_torepr functions internal
2022-01-12 17:33:58 +01:00
Mattias Wadman
a6bf62ce19 msgpack: Add decoder
Fixes #25
2022-01-10 19:33:09 +01:00
Mattias Wadman
af8e7efc61 bencode: Add decoder
Closes #63
2022-01-10 10:52:14 +01:00
siddik.acil
0cf486dcf1 elf: fix all-platforms naming typo 2022-01-08 13:42:08 +01:00
Mattias Wadman
c05fe732b6
Merge pull request #59 from wader/png-plte-trns-cleanup
png: Decode PLTE and tRNS chunks and cleanup syms a bit
2022-01-07 20:13:37 +01:00
Mattias Wadman
bfc977b810 png: Decode PLTE and tRNS chunks and cleanup syms a bit 2022-01-07 14:31:18 +01:00
Mattias Wadman
edd0ae198b tcp,flow: By default allow missing syn/ack for now
Is probably what you usually want
2022-01-07 12:02:38 +01:00
Mattias Wadman
628f0f403d bson: Add decoder
Only supports basic types for now
2022-01-06 00:51:37 +01:00
Mattias Wadman
681dbc26b7 elf,ar: Add ar decoder, improved elf decoder
Add x86, 386, arm64, arm v6, arm v7 linux elf test files
2022-01-05 22:05:09 +01:00
Mattias Wadman
af23eb8269 pcap: Don't fail if incl_len > spanlen
Documentation wrong? seems to happen in real pcap files
2022-01-03 21:27:14 +01:00
Mattias Wadman
b9aef39ed4 pcap,pcapng,bsd_loopback_frame: Add decoder, refactor link frame into a group
Also fix incorrect struct name s/UDPDatagramIn/UDPPayloadIn/
2022-01-03 21:06:27 +01:00
Mattias Wadman
b8671137dd matroska: Assert sane tag size only for strings
Fixes issue with big webm/matroska files
Can be ignored with force decode
2022-01-01 19:37:12 +01:00
Mattias Wadman
eb9698fce7 mp4,ctts: Seem more usable to treat sample count/offset as signed
Seems to different between specs
Modify avc.fqtest to generate a ctts box
2021-12-20 11:19:38 +01:00
Mattias Wadman
9f08af31df mpeg,aac: Factor out escape value decoding 2021-12-14 17:34:38 +01:00
Mattias Wadman
28a3b71bd4 mp4: Cleanup sample decode code 2021-12-12 15:25:57 +01:00
Mattias Wadman
91217e8210 tiff: Fix endian typo and cleanup todos 2021-12-12 11:26:00 +01:00
Mattias Wadman
07a2ebe4b7 tiff,fuzz: Fatal error on infinite ifd loops 2021-12-12 11:20:01 +01:00
Mattias Wadman
c149732182 mp4,trun,fuzz: Limit number of constant sample entries 2021-12-12 09:55:07 +01:00
Mattias Wadman
2b2320d411 pcap,flows: fuzz: Handle broken packets more nicely
Still need to understand how gopacket is suppose to handle broken things
2021-12-09 14:19:17 +01:00
Mattias Wadman
62785291fe mp4: fuzz: Make sure stsz has sane number of entries on constant sample size
Should be redone to not use an array
2021-12-09 13:00:10 +01:00
Mattias Wadman
f3480026bf sll2: fuzz: Limit address length to max 8 bytes 2021-12-09 10:38:34 +01:00
Mattias Wadman
6ed2e2e72e interp: dump: Indicate arrays using jq-syntax
Related to #16
2021-12-08 16:24:28 +01:00
Mattias Wadman
406263b485 mp4: Add comment about hdlr.component_name prefix byte 2021-12-08 10:45:10 +01:00
Mattias Wadman
c2131bb4a0 flac: Cleanup scalar usage and fix incorrect sample rates
Also split DisplayFormat into actual and sym
2021-12-08 00:39:52 +01:00
Mattias Wadman
1d7ace3899 pcap,pcapng,tcp: Use capture length not original length 2021-12-07 18:47:31 +01:00
Mattias Wadman
e91b22b324 matroska,ebml: Use scalar and require sane tag size 2021-12-07 15:51:46 +01:00
Mattias Wadman
45026ebf21 tar: Cleanup constant usage a bit more 2021-12-06 21:08:40 +01:00