## Supported formats [fq -rn -L . 'include "formats"; formats_table']: sh-start |Name |Description |Dependencies| |- |- |-| |[`aac_frame`](#aac_frame) |Advanced Audio Coding frame || |`adts` |Audio Data Transport Stream |`adts_frame`| |`adts_frame` |Audio Data Transport Stream frame |`aac_frame`| |`amf0` |Action Message Format 0 || |`apev2` |APEv2 metadata tag |`image`| |`ar` |Unix archive |`probe`| |[`asn1_ber`](#asn1_ber) |ASN1 BER (basic encoding rules, also CER and DER)|| |`av1_ccr` |AV1 Codec Configuration Record || |`av1_frame` |AV1 frame |`av1_obu`| |`av1_obu` |AV1 Open Bitstream Unit || |`avc_annexb` |H.264/AVC Annex B |`avc_nalu`| |[`avc_au`](#avc_au) |H.264/AVC Access Unit |`avc_nalu`| |`avc_dcr` |H.264/AVC Decoder Configuration Record |`avc_nalu`| |`avc_nalu` |H.264/AVC Network Access Layer Unit |`avc_sps` `avc_pps` `avc_sei`| |`avc_pps` |H.264/AVC Picture Parameter Set || |`avc_sei` |H.264/AVC Supplemental Enhancement Information || |`avc_sps` |H.264/AVC Sequence Parameter Set || |[`avro_ocf`](#avro_ocf) |Avro object container file || |[`bencode`](#bencode) |BitTorrent bencoding || |`bsd_loopback_frame` |BSD loopback frame |`inet_packet`| |[`bson`](#bson) |Binary JSON || |`bzip2` |bzip2 compression |`probe`| |[`cbor`](#cbor) |Concise Binary Object Representation || |`dns` |DNS packet || |`dns_tcp` |DNS packet (TCP) || |`elf` |Executable and Linkable Format || |`ether8023_frame` |Ethernet 802.3 frame |`inet_packet`| |`exif` |Exchangeable Image File Format || |`flac` |Free Lossless Audio Codec file |`flac_metadatablocks` `flac_frame`| |[`flac_frame`](#flac_frame) |FLAC frame || |`flac_metadatablock` |FLAC metadatablock |`flac_streaminfo` `flac_picture` `vorbis_comment`| |`flac_metadatablocks` |FLAC metadatablocks |`flac_metadatablock`| |`flac_picture` |FLAC metadatablock picture |`image`| |`flac_streaminfo` |FLAC streaminfo || |`gif` |Graphics Interchange Format || |`gzip` |gzip compression |`probe`| |`hevc_annexb` |H.265/HEVC Annex B |`hevc_nalu`| |[`hevc_au`](#hevc_au) |H.265/HEVC Access Unit |`hevc_nalu`| |`hevc_dcr` |H.265/HEVC Decoder Configuration Record |`hevc_nalu`| |`hevc_nalu` |H.265/HEVC Network Access Layer Unit |`hevc_vps` `hevc_pps` `hevc_sps`| |`hevc_pps` |H.265/HEVC Picture Parameter Set || |`hevc_sps` |H.265/HEVC Sequence Parameter Set || |`hevc_vps` |H.265/HEVC Video Parameter Set || |`icc_profile` |International Color Consortium profile || |`icmp` |Internet Control Message Protocol || |`icmpv6` |Internet Control Message Protocol v6 || |`id3v1` |ID3v1 metadata || |`id3v11` |ID3v1.1 metadata || |`id3v2` |ID3v2 metadata |`image`| |`ipv4_packet` |Internet protocol v4 packet |`ip_packet`| |`ipv6_packet` |Internet protocol v6 packet |`ip_packet`| |`jpeg` |Joint Photographic Experts Group file |`exif` `icc_profile`| |`json` |JSON || |[`macho`](#macho) |Mach-O macOS executable || |[`matroska`](#matroska) |Matroska file |`aac_frame` `av1_ccr` `av1_frame` `avc_au` `avc_dcr` `flac_frame` `flac_metadatablocks` `hevc_au` `hevc_dcr` `image` `mp3_frame` `mpeg_asc` `mpeg_pes_packet` `mpeg_spu` `opus_packet` `vorbis_packet` `vp8_frame` `vp9_cfm` `vp9_frame`| |[`mp3`](#mp3) |MP3 file |`id3v2` `id3v1` `id3v11` `apev2` `mp3_frame`| |`mp3_frame` |MPEG audio layer 3 frame |`xing`| |[`mp4`](#mp4) |ISOBMFF MPEG-4 part 12 and similar |`aac_frame` `av1_ccr` `av1_frame` `flac_frame` `flac_metadatablocks` `id3v2` `image` `jpeg` `mp3_frame` `avc_au` `avc_dcr` `mpeg_es` `hevc_au` `hevc_dcr` `mpeg_pes_packet` `opus_packet` `protobuf_widevine` `pssh_playready` `vorbis_packet` `vp9_frame` `vpx_ccr` `icc_profile`| |`mpeg_asc` |MPEG-4 Audio Specific Config || |`mpeg_es` |MPEG Elementary Stream |`mpeg_asc` `vorbis_packet`| |`mpeg_pes` |MPEG Packetized elementary stream |`mpeg_pes_packet` `mpeg_spu`| |`mpeg_pes_packet` |MPEG Packetized elementary stream packet || |`mpeg_spu` |Sub Picture Unit (DVD subtitle) || |`mpeg_ts` |MPEG Transport Stream || |[`msgpack`](#msgpack) |MessagePack || |`ogg` |OGG file |`ogg_page` `vorbis_packet` `opus_packet` `flac_metadatablock` `flac_frame`| |`ogg_page` |OGG page || |`opus_packet` |Opus packet |`vorbis_comment`| |`pcap` |PCAP packet capture |`link_frame` `tcp_stream` `ipv4_packet`| |`pcapng` |PCAPNG packet capture |`link_frame` `tcp_stream` `ipv4_packet`| |`png` |Portable Network Graphics file |`icc_profile` `exif`| |[`protobuf`](#protobuf) |Protobuf || |`protobuf_widevine` |Widevine protobuf |`protobuf`| |`pssh_playready` |PlayReady PSSH || |`raw` |Raw bits || |[`rtmp`](#rtmp) |Real-Time Messaging Protocol |`amf0` `mpeg_asc`| |`sll2_packet` |Linux cooked capture encapsulation v2 |`inet_packet`| |`sll_packet` |Linux cooked capture encapsulation |`inet_packet`| |`tar` |Tar archive |`probe`| |`tcp_segment` |Transmission control protocol segment || |`tiff` |Tag Image File Format |`icc_profile`| |`udp_datagram` |User datagram protocol |`udp_payload`| |`vorbis_comment` |Vorbis comment |`flac_picture`| |`vorbis_packet` |Vorbis packet |`vorbis_comment`| |`vp8_frame` |VP8 frame || |`vp9_cfm` |VP9 Codec Feature Metadata || |`vp9_frame` |VP9 frame || |`vpx_ccr` |VPX Codec Configuration Record || |`wav` |WAV file |`id3v2` `id3v1` `id3v11`| |`webp` |WebP image |`vp8_frame`| |`xing` |Xing header || |`zip` |ZIP archive |`probe`| |`image` |Group |`gif` `jpeg` `mp4` `png` `tiff` `webp`| |`inet_packet` |Group |`ipv4_packet` `ipv6_packet`| |`ip_packet` |Group |`icmp` `icmpv6` `tcp_segment` `udp_datagram`| |`link_frame` |Group |`bsd_loopback_frame` `ether8023_frame` `sll2_packet` `sll_packet`| |`probe` |Group |`adts` `ar` `avro_ocf` `bzip2` `elf` `flac` `gif` `gzip` `jpeg` `json` `macho` `matroska` `mp3` `mp4` `mpeg_ts` `ogg` `pcap` `pcapng` `png` `tar` `tiff` `wav` `webp` `zip`| |`tcp_stream` |Group |`dns` `rtmp`| |`udp_payload` |Group |`dns`| [#]: sh-end ## Global format options Currently the only global option is `force` and is used to ignore some format assertion errors. It can be used as a decode option or as a CLI `-o` option: ``` fq -d mp4 -o force=true file.mp4 fq -d raw 'mp4({force: true})' file.mp4 ``` ## Format details [fq -rn -L . 'include "formats"; formats_sections']: sh-start ### aac_frame #### Options |Name |Default|Description| |- |- |-| |`object_type`|1 |Audio object type| #### Examples Decode file using options ``` $ fq -d aac_frame -o object_type=1 file ``` Decode value as aac_frame ``` ... | aac_frame({object_type: 1}) ``` ### asn1_ber Supports decoding BER, CER and DER (X.690). - Currently no extra validation is done for CER and DER. - Does not support specifying a schema. - Supports `torepr` but without schema all sequences and sets will be arrays. #### Examples `frompem` and `topem` can be used to work with PEM format ``` $ fq -d raw 'frompem | asn1_ber | d' cert.pem ``` Can be used to decode nested parts ``` $ fq -d asn1_ber '.constructed[1].value | asn1_ber' file.ber ``` If schema is known and not complicated it can be reproduced ``` $ fq -d asn1_ber 'torepr as $r | ["version", "modulus", "private_exponent", "private_exponen", "prime1", "prime2", "exponent1", "exponent2", "coefficient"] | with_entries({key: .value, value: $r[.key]})' pkcs1.der ``` Supports `torepr` ``` $ fq -d asn1_ber torepr file ``` Supports `torepr` ``` ... | asn1_ber | torepr ``` #### References and links - https://www.itu.int/ITU-T/studygroups/com10/languages/X.690_1297.pdf - https://en.wikipedia.org/wiki/X.690 - https://letsencrypt.org/docs/a-warm-welcome-to-asn1-and-der/ - https://lapo.it/asn1js/ ### avc_au #### Options |Name |Default|Description| |- |- |-| |`length_size`|4 |Length value size| #### Examples Decode file using options ``` $ fq -d avc_au -o length_size=4 file ``` Decode value as avc_au ``` ... | avc_au({length_size: 4}) ``` ### avro_ocf Supports reading Avro Object Container Format (OCF) files based on the 1.11.0 specification. Capable of handling null, deflate, and snappy codecs for data compression. Limitations: - Schema does not support self-referential types, only built-in types. - Decimal logical types are not supported for decoding, will just be treated as their primitive type #### References and links - https://avro.apache.org/docs/current/spec.html#Object+Container+Files ### bencode #### Examples Supports `torepr` ``` $ fq -d bencode torepr file ``` Supports `torepr` ``` ... | bencode | torepr ``` #### References and links - https://bsonspec.org/spec.html ### bson #### Examples BSON as JSON ``` $ fq -d bson torepr file ``` Supports `torepr` ``` $ fq -d bson torepr file ``` Supports `torepr` ``` ... | bson | torepr ``` #### References and links - https://wiki.theory.org/BitTorrentSpecification#Bencoding ### cbor #### Examples Supports `torepr` ``` $ fq -d cbor torepr file ``` Supports `torepr` ``` ... | cbor | torepr ``` #### References and links - https://en.wikipedia.org/wiki/CBOR - https://www.rfc-editor.org/rfc/rfc8949.html ### flac_frame #### Options |Name |Default|Description| |- |- |-| |`bits_per_sample`|16 |Bits per sample| #### Examples Decode file using options ``` $ fq -d flac_frame -o bits_per_sample=16 file ``` Decode value as flac_frame ``` ... | flac_frame({bits_per_sample: 16}) ``` ### hevc_au #### Options |Name |Default|Description| |- |- |-| |`length_size`|4 |Length value size| #### Examples Decode file using options ``` $ fq -d hevc_au -o length_size=4 file ``` Decode value as hevc_au ``` ... | hevc_au({length_size: 4}) ``` ### macho Supports decoding vanilla and FAT Mach-O binaries. #### Examples Select 64bit load segments ``` $ fq '.load_commands[] | select(.cmd=="segment_64")' file ``` #### References and links - https://github.com/aidansteele/osx-abi-macho-file-format-reference ### matroska #### Examples Lookup element decode value using `matroska_path` ``` ... | matroska_path(".Segment.Tracks[0)" ``` Return `matroska_path` string for a box decode value ``` ... | grep_by(.id == "Tracks") | matroska_path ``` #### References and links - https://tools.ietf.org/html/draft-ietf-cellar-ebml-00 - https://matroska.org/technical/specs/index.html - https://www.matroska.org/technical/basics.html - https://www.matroska.org/technical/codec_specs.html - https://wiki.xiph.org/MatroskaOpus ### mp3 #### Options |Name |Default|Description| |- |- |-| |`max_sync_seek` |32768 |Max byte distance to next sync| |`max_unique_header_configs`|5 |Max number of unique frame header configs allowed| #### Examples Decode file using options ``` $ fq -d mp3 -o max_sync_seek=32768 -o max_unique_header_configs=5 file ``` Decode value as mp3 ``` ... | mp3({max_sync_seek: 32768, max_unique_header_configs: 5}) ``` ### mp4 Support `mp4_path` #### Options |Name |Default|Description| |- |- |-| |`allow_truncated`|false |Allow box to be truncated| |`decode_samples` |true |Decode supported media samples| #### Examples Lookup box decode value using `mp4_path` ``` ... | mp4_path(".moov.trak[1]") ``` Return `mp4_path` string for a box decode value ``` ... | grep_by(.type == "trak") | mp4_path ``` Decode file using options ``` $ fq -d mp4 -o allow_truncated=false -o decode_samples=true file ``` Decode value as mp4 ``` ... | mp4({allow_truncated: false, decode_samples: true}) ``` #### References and links - [ISO/IEC base media file format (MPEG-4 Part 12)](https://en.wikipedia.org/wiki/ISO/IEC_base_media_file_format) - [Quicktime file format](https://developer.apple.com/standards/qtff-2001.pdf) ### msgpack #### Examples Supports `torepr` ``` $ fq -d msgpack torepr file ``` Supports `torepr` ``` ... | msgpack | torepr ``` #### References and links - https://github.com/msgpack/msgpack/blob/master/spec.md ### protobuf #### Examples Can be used to decode sub messages ``` $ fq -d protobuf '.fields[6].wire_value | protobuf | d' ``` #### References and links - https://developers.google.com/protocol-buffers/docs/encoding ### rtmp Current only supports plain RTMP (not RTMPT or encrypted variants etc) with AMF0 (not AMF3). #### References and links - https://rtmp.veriskope.com/docs/spec/ - https://rtmp.veriskope.com/pdf/video_file_format_spec_v10.pdf [#]: sh-end ## Dependency graph ![alt text](formats.svg "Format diagram")