wader/fq
1
1
Fork 0
mirror of https://github.com/wader/fq.git synced 2024-12-25 06:12:30 +03:00
Code Issues Projects Releases Wiki Activity
67873ca944
fq/doc/formats.md
Mattias Wadman 1aaaefb0ac wav,bencode,mpeg_ps_packet,id3v1: Random fixes 
wav: Rest of file should description not symbolic value
bencode: Remove redundant torepr example
mpeg_ps_packet,id3v1: Remove whitespace in symbolic value
2022-05-05 19:45:05 +02:00

21 KiB
Raw Blame History

Supported formats

Name Description Dependencies
aac_frame Advanced Audio Coding frame
adts Audio Data Transport Stream adts_frame
adts_frame Audio Data Transport Stream frame aac_frame
amf0 Action Message Format 0
apev2 APEv2 metadata tag image
ar Unix archive probe
asn1_ber ASN1 BER (basic encoding rules, also CER and DER)
av1_ccr AV1 Codec Configuration Record
av1_frame AV1 frame av1_obu
av1_obu AV1 Open Bitstream Unit
avc_annexb H.264/AVC Annex B avc_nalu
avc_au H.264/AVC Access Unit avc_nalu
avc_dcr H.264/AVC Decoder Configuration Record avc_nalu
avc_nalu H.264/AVC Network Access Layer Unit avc_sps avc_pps avc_sei
avc_pps H.264/AVC Picture Parameter Set
avc_sei H.264/AVC Supplemental Enhancement Information
avc_sps H.264/AVC Sequence Parameter Set
avro_ocf Avro object container file
bencode BitTorrent bencoding
bsd_loopback_frame BSD loopback frame inet_packet
bson Binary JSON
bzip2 bzip2 compression probe
cbor Concise Binary Object Representation
dns DNS packet
dns_tcp DNS packet (TCP)
elf Executable and Linkable Format
ether8023_frame Ethernet 802.3 frame inet_packet
exif Exchangeable Image File Format
flac Free Lossless Audio Codec file flac_metadatablocks flac_frame
flac_frame FLAC frame
flac_metadatablock FLAC metadatablock flac_streaminfo flac_picture vorbis_comment
flac_metadatablocks FLAC metadatablocks flac_metadatablock
flac_picture FLAC metadatablock picture image
flac_streaminfo FLAC streaminfo
gif Graphics Interchange Format
gzip gzip compression probe
hevc_annexb H.265/HEVC Annex B hevc_nalu
hevc_au H.265/HEVC Access Unit hevc_nalu
hevc_dcr H.265/HEVC Decoder Configuration Record hevc_nalu
hevc_nalu H.265/HEVC Network Access Layer Unit hevc_vps hevc_pps hevc_sps
hevc_pps H.265/HEVC Picture Parameter Set
hevc_sps H.265/HEVC Sequence Parameter Set
hevc_vps H.265/HEVC Video Parameter Set
icc_profile International Color Consortium profile
icmp Internet Control Message Protocol
icmpv6 Internet Control Message Protocol v6
id3v1 ID3v1 metadata
id3v11 ID3v1.1 metadata
id3v2 ID3v2 metadata image
ipv4_packet Internet protocol v4 packet ip_packet
ipv6_packet Internet protocol v6 packet ip_packet
jpeg Joint Photographic Experts Group file exif icc_profile
json JSON
macho Mach-O macOS executable
matroska Matroska file aac_frame av1_ccr av1_frame avc_au avc_dcr flac_frame flac_metadatablocks hevc_au hevc_dcr image mp3_frame mpeg_asc mpeg_pes_packet mpeg_spu opus_packet vorbis_packet vp8_frame vp9_cfm vp9_frame
mp3 MP3 file id3v2 id3v1 id3v11 apev2 mp3_frame
mp3_frame MPEG audio layer 3 frame xing
mp4 ISOBMFF MPEG-4 part 12 and similar aac_frame av1_ccr av1_frame flac_frame flac_metadatablocks id3v2 image jpeg mp3_frame avc_au avc_dcr mpeg_es hevc_au hevc_dcr mpeg_pes_packet opus_packet protobuf_widevine pssh_playready vorbis_packet vp9_frame vpx_ccr icc_profile
mpeg_asc MPEG-4 Audio Specific Config
mpeg_es MPEG Elementary Stream mpeg_asc vorbis_packet
mpeg_pes MPEG Packetized elementary stream mpeg_pes_packet mpeg_spu
mpeg_pes_packet MPEG Packetized elementary stream packet
mpeg_spu Sub Picture Unit (DVD subtitle)
mpeg_ts MPEG Transport Stream
msgpack MessagePack
ogg OGG file ogg_page vorbis_packet opus_packet flac_metadatablock flac_frame
ogg_page OGG page
opus_packet Opus packet vorbis_comment
pcap PCAP packet capture link_frame tcp_stream ipv4_packet
pcapng PCAPNG packet capture link_frame tcp_stream ipv4_packet
png Portable Network Graphics file icc_profile exif
protobuf Protobuf
protobuf_widevine Widevine protobuf protobuf
pssh_playready PlayReady PSSH
raw Raw bits
rtmp Real-Time Messaging Protocol amf0 mpeg_asc
sll2_packet Linux cooked capture encapsulation v2 inet_packet
sll_packet Linux cooked capture encapsulation inet_packet
tar Tar archive probe
tcp_segment Transmission control protocol segment
tiff Tag Image File Format icc_profile
udp_datagram User datagram protocol udp_payload
vorbis_comment Vorbis comment flac_picture
vorbis_packet Vorbis packet vorbis_comment
vp8_frame VP8 frame
vp9_cfm VP9 Codec Feature Metadata
vp9_frame VP9 frame
vpx_ccr VPX Codec Configuration Record
wav WAV file id3v2 id3v1 id3v11
webp WebP image vp8_frame
xing Xing header
zip ZIP archive probe
image Group gif jpeg mp4 png tiff webp
inet_packet Group ipv4_packet ipv6_packet
ip_packet Group icmp icmpv6 tcp_segment udp_datagram
link_frame Group bsd_loopback_frame ether8023_frame sll2_packet sll_packet
probe Group adts ar avro_ocf bzip2 elf flac gif gzip jpeg json macho matroska mp3 mp4 mpeg_ts ogg pcap pcapng png tar tiff wav webp zip
tcp_stream Group dns rtmp
udp_payload Group dns

Global format options

Currently the only global option is force and is used to ignore some format assertion errors. It can be used as a decode option or as a CLI -o option:

fq -d mp4 -o force=true file.mp4
fq -d raw 'mp4({force: true})' file.mp4

Format details

aac_frame

Options

Name Default Description
object_type 1 Audio object type

Examples

Decode file using options

$ fq -d aac_frame -o object_type=1 file

Decode value as aac_frame

... | aac_frame({object_type: 1})

asn1_ber

Supports decoding BER, CER and DER (X.690).

  • Currently no extra validation is done for CER and DER.
  • Does not support specifying a schema.
  • Supports torepr but without schema all sequences and sets will be arrays.

Examples

frompem and topem can be used to work with PEM format

$ fq -d raw 'frompem | asn1_ber | d' cert.pem

Can be used to decode nested parts

$ fq -d asn1_ber '.constructed[1].value | asn1_ber' file.ber

If schema is known and not complicated it can be reproduced

$ fq -d asn1_ber 'torepr as $r | ["version", "modulus", "private_exponent", "private_exponen", "prime1", "prime2", "exponent1", "exponent2", "coefficient"] | with_entries({key: .value, value: $r[.key]})' pkcs1.der

Supports torepr

$ fq -d asn1_ber torepr file

Supports torepr

... | asn1_ber | torepr

References and links

avc_au

Options

Name Default Description
length_size 4 Length value size

Examples

Decode file using options

$ fq -d avc_au -o length_size=4 file

Decode value as avc_au

... | avc_au({length_size: 4})

avro_ocf

Supports reading Avro Object Container Format (OCF) files based on the 1.11.0 specification.

Capable of handling null, deflate, and snappy codecs for data compression.

Limitations:

  • Schema does not support self-referential types, only built-in types.
  • Decimal logical types are not supported for decoding, will just be treated as their primitive type

References and links

bencode

Examples

Supports torepr

$ fq -d bencode torepr file

Supports torepr

... | bencode | torepr

References and links

bson

Examples

BSON as JSON

$ fq -d bson torepr file

Supports torepr

$ fq -d bson torepr file

Supports torepr

... | bson | torepr

References and links

cbor

Examples

Supports torepr

$ fq -d cbor torepr file

Supports torepr

... | cbor | torepr

References and links

flac_frame

Options

Name Default Description
bits_per_sample 16 Bits per sample

Examples

Decode file using options

$ fq -d flac_frame -o bits_per_sample=16 file

Decode value as flac_frame

... | flac_frame({bits_per_sample: 16})

hevc_au

Options

Name Default Description
length_size 4 Length value size

Examples

Decode file using options

$ fq -d hevc_au -o length_size=4 file

Decode value as hevc_au

... | hevc_au({length_size: 4})

macho

Supports decoding vanilla and FAT Mach-O binaries.

Examples

Select 64bit load segments

$ fq '.load_commands[] | select(.cmd=="segment_64")' file

References and links

matroska

Examples

Lookup element decode value using matroska_path

... | matroska_path(".Segment.Tracks[0)"

Return matroska_path string for a box decode value

... | grep_by(.id == "Tracks") | matroska_path

References and links

mp3

Options

Name Default Description
max_sync_seek 32768 Max byte distance to next sync
max_unique_header_configs 5 Max number of unique frame header configs allowed

Examples

Decode file using options

$ fq -d mp3 -o max_sync_seek=32768 -o max_unique_header_configs=5 file

Decode value as mp3

... | mp3({max_sync_seek: 32768, max_unique_header_configs: 5})

mp4

Support mp4_path

Options

Name Default Description
allow_truncated false Allow box to be truncated
decode_samples true Decode supported media samples

Examples

Lookup box decode value using mp4_path

... | mp4_path(".moov.trak[1]")

Return mp4_path string for a box decode value

... | grep_by(.type == "trak") | mp4_path

Decode file using options

$ fq -d mp4 -o allow_truncated=false -o decode_samples=true file

Decode value as mp4

... | mp4({allow_truncated: false, decode_samples: true})

References and links

msgpack

Examples

Supports torepr

$ fq -d msgpack torepr file

Supports torepr

... | msgpack | torepr

References and links

protobuf

Examples

Can be used to decode sub messages

$ fq -d protobuf '.fields[6].wire_value | protobuf | d'

References and links

rtmp

Current only supports plain RTMP (not RTMPT or encrypted variants etc) with AMF0 (not AMF3).

References and links

Dependency graph

alt text