wader/fq
1
1
Fork 0
mirror of https://github.com/wader/fq.git synced 2024-11-28 11:42:50 +03:00
Code Issues Projects Releases Wiki Activity
948 Commits 33 Branches 23 Tags 59 MiB
8228ecaeee
Commit Graph

309 Commits

Author SHA1 Message Date
Mattias Wadman
af23eb8269 pcap: Don't fail if incl_len > spanlen 
Documentation wrong? seems to happen in real pcap files
 2022-01-03 21:27:14 +01:00
Mattias Wadman
b9aef39ed4 pcap,pcapng,bsd_loopback_frame: Add decoder, refactor link frame into a group 
Also fix incorrect struct name s/UDPDatagramIn/UDPPayloadIn/
 2022-01-03 21:06:27 +01:00
Mattias Wadman
b8671137dd matroska: Assert sane tag size only for strings 
Fixes issue with big webm/matroska files
Can be ignored with force decode
 2022-01-01 19:37:12 +01:00
Xentripetal
68e85a2ddf Use existing scalar description helper 2021-12-29 17:01:40 -06:00
Xentripetal
07ddf36f1c Cleanup for linting 2021-12-29 16:46:56 -06:00
Xentripetal
4b809a73f6 Change avro codec to funcs 2021-12-29 16:12:36 -06:00
Xentripetal
7345b8c7c2 Cleanup 2021-12-28 13:09:54 -06:00
Xentripetal
7a8e3ca26e Hook into registry, add codecs 2021-12-28 13:05:10 -06:00
Xentripetal
d6ca48182e initial work for avro OCF files 2021-12-28 13:01:53 -06:00
Mattias Wadman
eb9698fce7 mp4,ctts: Seem more usable to treat sample count/offset as signed 
Seems to different between specs
Modify avc.fqtest to generate a ctts box
 2021-12-20 11:19:38 +01:00
Mattias Wadman
9f08af31df mpeg,aac: Factor out escape value decoding 2021-12-14 17:34:38 +01:00
Mattias Wadman
28a3b71bd4 mp4: Cleanup sample decode code 2021-12-12 15:25:57 +01:00
Mattias Wadman
91217e8210 tiff: Fix endian typo and cleanup todos 2021-12-12 11:26:00 +01:00
Mattias Wadman
07a2ebe4b7 tiff,fuzz: Fatal error on infinite ifd loops 2021-12-12 11:20:01 +01:00
Mattias Wadman
c149732182 mp4,trun,fuzz: Limit number of constant sample entries 2021-12-12 09:55:07 +01:00
Mattias Wadman
2b2320d411 pcap,flows: fuzz: Handle broken packets more nicely 
Still need to understand how gopacket is suppose to handle broken things
 2021-12-09 14:19:17 +01:00
Mattias Wadman
62785291fe mp4: fuzz: Make sure stsz has sane number of entries on constant sample size 
Should be redone to not use an array
 2021-12-09 13:00:10 +01:00
Mattias Wadman
f3480026bf sll2: fuzz: Limit address length to max 8 bytes 2021-12-09 10:38:34 +01:00
Mattias Wadman
6ed2e2e72e interp: dump: Indicate arrays using jq-syntax 
Related to #16
 2021-12-08 16:24:28 +01:00
Mattias Wadman
406263b485 mp4: Add comment about hdlr.component_name prefix byte 2021-12-08 10:45:10 +01:00
Mattias Wadman
c2131bb4a0 flac: Cleanup scalar usage and fix incorrect sample rates 
Also split DisplayFormat into actual and sym
 2021-12-08 00:39:52 +01:00
Mattias Wadman
1d7ace3899 pcap,pcapng,tcp: Use capture length not original length 2021-12-07 18:47:31 +01:00
Mattias Wadman
e91b22b324 matroska,ebml: Use scalar and require sane tag size 2021-12-07 15:51:46 +01:00
Mattias Wadman
45026ebf21 tar: Cleanup constant usage a bit more 2021-12-06 21:08:40 +01:00
Mattias Wadman
b525d0b3c8 pcap: fuzz: Skip ssl2 packet if too short 
Should report error somehow?
 2021-12-06 19:05:04 +01:00
Mattias Wadman
38509683a7 udp: Use proper udp payload format var name 2021-12-06 15:42:49 +01:00
Mattias Wadman
3601fe3bcb gzip: fuzz: Don't uncompress on unknown compress method (nil create reader fn) 2021-12-06 15:41:39 +01:00
Mattias Wadman
91cc6d80f7 tar: Fix size decode regression after cleanup 2021-12-06 14:15:54 +01:00
Mattias Wadman
e260830454 webp: Cleanup endian usage 2021-12-04 19:15:54 +01:00
Mattias Wadman
dd883b35bd wav: Cleanup endian usage 2021-12-04 19:14:07 +01:00
Mattias Wadman
57e9f418c8 vorbis_packet: Cleanup endian usage 2021-12-04 19:12:24 +01:00
Mattias Wadman
dc1aea3030 opus: Cleanup endian usage and fix incorrect preskip decode 2021-12-04 19:12:24 +01:00
Mattias Wadman
6a8d77b05c vorbis_comment: Cleanup endian usage and naming a bit 2021-12-04 19:12:24 +01:00
Mattias Wadman
eb4718fbdd tar: Cleanup api usage 2021-12-04 19:05:26 +01:00
Mattias Wadman
d48ebc12ee decode: Simplify Compound.Children 
Not a array reference anymore but instead pass around Compound reference
 2021-12-03 00:06:11 +01:00
Mattias Wadman
2fc0a71a47 decode: Refactor scalar usage 
Move scalar into own package.
Split scalar code into decode related scalar code (that reads etc) and
scalar code that just transform the scalar value.
Use a scalar.Mapper interface instead of just a function.
Make mappers, assert and validat impement the interface.
 2021-12-02 17:39:26 +01:00
Mattias Wadman
f801cc0af7 decode: Rename s/FieldTryFormat/TryFieldFormat for consistency 2021-11-30 15:29:41 +01:00
Mattias Wadman
c0eebcc23f format: Remove unused ProtoBufType 2021-11-30 13:13:14 +01:00
Mattias Wadman
7f7698601f ogg: Cleanup bitio in format out, maybe later 2021-11-30 13:12:14 +01:00
Mattias Wadman
f55b1af6ac inet: Add tcp and ipv4 reassembly 
Also add tcp_stream and udp_payload to decode content
 2021-11-29 18:42:18 +01:00
Mattias Wadman
be0fdbe793 vp9: Add profile and fix reserved_zero field collision 2021-11-29 16:14:59 +01:00
Mattias Wadman
db586eb3f9 mp4,matroska: Add *_path/0 variant that uses format_root 2021-11-29 13:19:23 +01:00
Mattias Wadman
c8fad57df7 tiff: Fix reading of mluc tags with multiple records 2021-11-29 11:55:25 +01:00
Mattias Wadman
646f902ca7 vpx_ccr: Add color names 2021-11-29 00:37:22 +01:00
Mattias Wadman
3bf1a57cfa avc: Cleanup and add color names etc 2021-11-28 17:29:56 +01:00
Mattias Wadman
7b7faaf02b pcap: Add pcap, pcapng, ether8023, ipv4, udp, udp 2021-11-24 18:13:00 +01:00
Mattias Wadman
9d116df799 decode: Rework use of TryFieldReaderRangeFormat 
Handle failure better
 2021-11-24 15:25:27 +01:00
Mattias Wadman
4f0bf9228f flac: Make md5_calculated be a buffer 2021-11-24 12:31:42 +01:00
Mattias Wadman
4f8d03731d format,decode: Some crc and endian refactor 2021-11-24 12:01:01 +01:00
Mattias Wadman
fc76907c07 png: Add proper color type 2021-11-23 17:04:21 +01:00
Mattias Wadman
2af08da9f7 flac: Fix block_size regression 2021-11-23 16:59:50 +01:00
Mattias Wadman
e47888e24c mp4: Fix 64bit size regression 2021-11-23 11:24:09 +01:00
Mattias Wadman
e6cb708d38 mpeg: Nicer sym and description 2021-11-22 17:02:07 +01:00
Mattias Wadman
2e71fa1380 mp4: Add smhd box 2021-11-22 13:26:32 +01:00
Mattias Wadman
27e477061e apev2: Fatal if > 1000 tags 2021-11-22 01:45:30 +01:00
Mattias Wadman
fdb811ec54 gojqextra,decode: Add generic lazy JQValue 2021-11-22 01:23:59 +01:00
Mattias Wadman
5ad048daf4 tar: Fix 0 trim regression 2021-11-22 00:45:13 +01:00
Mattias Wadman
aab32cf2db gojqextra,interp: Add lazy string to speed usage of decode value buffer where string is not used 2021-11-21 21:27:44 +01:00
Mattias Wadman
ee611a489a gzip,bzip2: Calculate CRC 2021-11-21 20:13:42 +01:00
Mattias Wadman
606c0b67ae format: Add vorbis-comment-picture test, add .gitignore and cleanup 2021-11-21 19:32:52 +01:00
Mattias Wadman
f40320b04c decode: Remove D.Scalar* and add d.(Try)FieldScala*Fn instead 
Idea is scalar fn should not read
 2021-11-21 13:08:18 +01:00
Mattias Wadman
046f2fd965 mp3: Don't allow more than 64k between frames 
Should rethink this
 2021-11-21 12:02:40 +01:00
Mattias Wadman
9029143acc zip: Fix nested decode for none compress 2021-11-21 12:01:09 +01:00
Mattias Wadman
0480a2f722 decode: Some format decode and sub buffer work 2021-11-20 18:56:59 +01:00
Mattias Wadman
d1b514edeb format: Some claeanup 2021-11-19 16:44:06 +01:00
Mattias Wadman
527f917698 mp3: Error if > 5 unique header configs 
Hopefully make it less likely to missprobe
 2021-11-19 16:22:18 +01:00
Mattias Wadman
278e909a2f mp4: Add tapt, prof, enof and clap boxes 2021-11-19 15:29:49 +01:00
Mattias Wadman
d838d2f613 zip: Add format decoder 2021-11-19 00:22:26 +01:00
Mattias Wadman
5344c7e3de icc: Add mluc support 2021-11-18 11:22:07 +01:00
Mattias Wadman
ffb5adff1d registry: Move to pkg/registry, feels better 2021-11-18 01:22:21 +01:00
Mattias Wadman
7af191d974 decode: Move io helper into *D 2021-11-18 01:17:41 +01:00
Mattias Wadman
6fba1a8125 decode: Fix bitbuf root handling a bit 2021-11-18 01:17:15 +01:00
Mattias Wadman
c083a9e475 decode: Fix MapRawToScalar regression 2021-11-17 17:14:19 +01:00
Mattias Wadman
986d5ecc50 decode: Move registry package to decode/registry and add a format group type 2021-11-17 16:59:56 +01:00
Mattias Wadman
3cea849101 dns: Cleanup a bit 2021-11-17 16:30:01 +01:00
Mattias Wadman
5d98a6949a decode: Refactor Error/Fatal into printf functions 2021-11-17 16:26:13 +01:00
Mattias Wadman
ede2e77975 decode: Nicer scalar template and add doc 2021-11-17 16:13:10 +01:00
Mattias Wadman
b66ed32614 decode,interp: Make fuzzing work again and cleanup fatal/error code 2021-11-16 17:11:26 +01:00
Mattias Wadman
f9f866000e interp,decode: Add force option to ignore asserts 2021-11-16 13:03:56 +01:00
Mattias Wadman
1b32b42f93 decode: Major decode API refactor 
Generate more code
More generic and comfortable API
Improve and Update format decoder to new API
Add some more format tests
 2021-11-15 21:12:07 +01:00
Mattias Wadman
cc5f405554 interp,format: Update tests after decode refactor/tosym/toactual 2021-11-05 17:29:23 +01:00
Mattias Wadman
6a15625587 interp,decode: Refactor out Scalar from Value and merge Array/Struct into Compound 
Also add tosym, toactual
 2021-11-05 17:29:22 +01:00
Mattias Wadman
13fae09172 interp: Don't print context cancel 2021-11-01 13:01:05 +01:00
Mattias Wadman
67898cb44a interp: Cleanup, use BufferRange for _open, progress for all decode 2021-10-30 01:17:14 +02:00
Mattias Wadman
88eade9ab9 ogg: Add flac support 
Refactor flac_streaminfo and flac_metadata from flac_metadatablocks
Some deocde bitbuf api cleanup
 2021-10-29 18:30:16 +02:00
Mattias Wadman
c1d9b4d2d5 flac_metadatablocks: typ > 127 can't happen, add app id 2021-10-23 23:06:03 +02:00
Mattias Wadman
bc9951cc59 flac_frame: Fail if trying to decode outside block size 2021-10-20 13:55:34 +02:00
Mattias Wadman
4af5739712 interp: Rework string/buffer for decode values 2021-10-20 01:31:59 +02:00
Mattias Wadman
49d2e617f9 tar: Unbreak num parsing and add test 2021-10-19 22:36:03 +02:00
Mattias Wadman
344f628403 flac: Calculate correct md5 when total samples count is zero 2021-10-19 18:05:22 +02:00
Mattias Wadman
1f26d4f233 flac_frame: Correctly read escaped samples and also a bit less allocations 2021-10-19 17:27:05 +02:00
Mattias Wadman
ce044baf7e flac: Cleanup 2021-10-19 15:44:15 +02:00
Mattias Wadman
5052bae18e decode,interp: Refactor to allow decode/fillgap a range 2021-10-19 14:38:11 +02:00
Mattias Wadman
2b35d28f3b flac_picture,mp4: avif images and data fallback is image format fails 2021-10-19 01:58:07 +02:00
Mattias Wadman
6ff5acae03 gif: Support GIF87a 2021-10-19 01:53:37 +02:00
Mattias Wadman
b643e22877 flac_frame: Support non-8 bit align sample size 2021-10-19 01:52:59 +02:00
Mattias Wadman
ec97eca054 format: Split default.go into format.go and shared.go 2021-10-18 11:51:50 +02:00
Mattias Wadman
984ba1aa43 flac_metadatablocks: type >= 127 is invalid 2021-10-13 21:40:20 +02:00
Mattias Wadman
509b8f8c50 flac_picture: Add picture_type names 2021-10-13 21:34:14 +02:00
Mattias Wadman
691688022f fqtest: Refactor our script part to own package 
Hopefully can be reused for docs generation
 2021-10-06 18:49:03 +02:00
Mattias Wadman
f9622c2dc7 vorbis_comment: Fix field name typo 2021-10-06 12:01:07 +02:00
Mattias Wadman
761c411ca9 exif: Add note about JPEGInterchangeFormat 2021-10-01 17:08:16 +02:00
Mattias Wadman
723542aea2 flac_frame: Make utf8Uint reader more correct and robust 2021-09-29 11:53:14 +02:00
Mattias Wadman
aa38ccf0e1 format,interp: Use MustGroup and add probe order test 2021-09-28 13:08:22 +02:00
Mattias Wadman
8a4f66f616 mp3: Probe order after formwats with raw samples and similar sync headers 2021-09-28 13:07:10 +02:00
Mattias Wadman
577c0f5315 format: Add panic if register after resolve 2021-09-28 13:04:51 +02:00
Mattias Wadman
3ff0c9b5e0 lint: Enable errcheck adnd revive 2021-09-27 11:01:14 +02:00
Mattias Wadman
2daa738de8 flac_frame: Use d.Invalid for posssible errors 2021-09-24 16:26:17 +02:00
Mattias Wadman
3e7e133047 interp: Move formats func def to jq 2021-09-21 16:42:35 +02:00
Mattias Wadman
ed21f36b23 mp3_frame: Rename samples_per_frame to sample_count 2021-09-20 18:54:20 +02:00
Mattias Wadman
d4142b875c decode,png: Add FieldFormatReaderLen, refactor out zlib to format 2021-09-19 20:49:15 +02:00
Mattias Wadman
3db11d3fa8 decode: Add UTF8Fn functions and trim some null terminated strings 2021-09-17 22:13:35 +02:00
Mattias Wadman
589207da20 mp4: Use descriptor field for all descriptor boxes 2021-09-17 15:51:37 +02:00
Mattias Wadman
797bd4d652 flac: Refactor flac_metadatablock into flac_metadatablocks 
metadatablocks are not really used alone
 2021-09-17 15:46:13 +02:00
Mattias Wadman
ee972f490d dump: Add ascii header 2021-09-16 22:40:03 +02:00
Mattias Wadman
6ee7977c53 decode,format: Allow root array 2021-09-16 16:27:47 +02:00
Mattias Wadman
473b2243c9 decode: Simplify and move format arg into DecodeOptions 2021-09-16 15:29:11 +02:00
Mattias Wadman
c7416e6dcb decode, interp: More buffer reuse 2021-09-16 12:24:53 +02:00
Mattias Wadman
777191f685 avc: Correct sign expgolomb decode 2021-09-15 00:32:00 +02:00
Mattias Wadman
13d5cbd81b lint: Remove usused nolint, should somehow tell about decode.Copy 2021-09-14 19:00:05 +02:00
Mattias Wadman
776a6b3136 decode: Reuse read buffer per decode to speed things up 2021-09-14 18:54:59 +02:00
Mattias Wadman
798141a0b0 format: Cleanup comments 2021-09-14 17:01:25 +02:00
Mattias Wadman
1ccab2da1e apev2: Add test 2021-09-14 17:00:45 +02:00
Mattias Wadman
9ac17bd0e3 mp4: Add comment about future truncate to size option 2021-09-14 14:35:13 +02:00
Mattias Wadman
57a120730b id3v2: Support GEOB tags 2021-09-14 12:55:20 +02:00
Mattias Wadman
e104748a5f mp3: Continue try find frames on error 2021-09-14 12:54:57 +02:00
Mattias Wadman
1325e5c19e mp3: Cleanup comments 2021-09-12 13:08:56 +02:00
Mattias Wadman
4344b621da mp3: Only look for supported mp3 sync headers 2021-09-12 13:08:55 +02:00
Mattias Wadman
da386ea282 mp3: Be more relaxed with zero padding, just warn 2021-09-12 13:08:55 +02:00
Mattias Wadman
962d84dc68 funcs: Make intdiv truncate to int 2021-09-12 13:08:55 +02:00
Mattias Wadman
48517c7c60 cli: Unbreak part of completion 2021-09-12 13:08:55 +02:00
Mattias Wadman
f8e5944f91 dump,json: Properly figure if compound or not 2021-09-12 13:08:55 +02:00
Mattias Wadman
a49e924d5f decode: Use golang.org/x/text for text decoding 2021-09-12 13:08:55 +02:00
Mattias Wadman
c4a31205c0 mp3,README: some cleanup 2021-09-12 13:08:55 +02:00
Mattias Wadman
161dcaffd4 mp4: Better fragmented mp4 support 2021-09-12 13:08:54 +02:00
Mattias Wadman
08018825cb mp4: Fix field name typo for sample_composition_time_offsets_present 2021-09-12 13:08:54 +02:00
Mattias Wadman
c7d45ff1e0 mp4: add pssh_playready format 2021-09-12 13:08:54 +02:00
Mattias Wadman
4242bf6013 *_annexb: Refactor into avc/hevc_annexb 2021-09-12 13:08:54 +02:00
Mattias Wadman
9f55b6e619 decode: Refactor and add symbol struct and mapping 2021-09-12 13:08:53 +02:00
Mattias Wadman
f4b11b4be2 mpeg_annexb: Add format 2021-09-12 13:08:53 +02:00
Mattias Wadman
45afbe6201 wip 2021-09-12 13:08:53 +02:00
Mattias Wadman
d23edaa4a5 fq: Rename bits/0 bytes/0 to tobits tobytes, remove string/0 2021-09-12 13:08:53 +02:00
Mattias Wadman
f600f2e54f build: Require go 1.17 2021-09-12 13:08:52 +02:00
Mattias Wadman
59e7fd4e94 icc_profile: Clamp alignment to end instead of check last tag (might not be last in buffer) 2021-09-12 13:08:52 +02:00
Mattias Wadman
a5b802b288 icc_profile: Correct alignment byte count 2021-09-12 13:08:52 +02:00
Mattias Wadman
50d00d823d icc_profile: Add element alignment bytes field 2021-09-12 13:08:51 +02:00
Mattias Wadman
4b48828b32 format: Rename source file to match format name 2021-09-12 13:08:51 +02:00
Mattias Wadman
f322e78e8e mp4: More _time decoding 2021-09-12 13:08:51 +02:00
Mattias Wadman
f1507f7f65 mod: Use proper path and dont use replace 2021-09-12 13:08:50 +02:00
Mattias Wadman
c155c89ba3 decode: Rename format *Decode to *Format 2021-09-12 13:08:50 +02:00
Mattias Wadman
8dc0f06b62 cli: Add completion tests 2021-09-12 13:08:49 +02:00
Mattias Wadman
60df04b098 mp3_frame: Only supports layer 3, fixes some probe issues 2021-09-12 13:08:49 +02:00
Mattias Wadman
6b8d26c0de mp4: Properly decode tfra 2021-09-12 13:08:49 +02:00
Mattias Wadman
f2c1327005 mp4: Use ISOBMFF naming and some more tests 2021-09-12 13:08:49 +02:00
Mattias Wadman
3693667df4 Start of configurable json bit buffer formats 2021-09-12 13:08:49 +02:00
Mattias Wadman
27095b5a29 es: Decode MPEG ASC if audio stream 2021-09-12 13:08:48 +02:00
Mattias Wadman
6a1fa045e7 Decode hvc1 as hevc samplesa also 2021-09-12 13:08:48 +02:00
Mattias Wadman
565f18df76 Correct avc_au format variable name 2021-09-12 13:08:48 +02:00
Mattias Wadman
970465996c Init 2021-09-12 13:08:42 +02:00