## Supported formats [./formats_table.sh]: sh-start |Name |Description |Dependencies| |- |- |-| |`aac_frame` |Advanced Audio Coding frame || |`adts` |Audio Data Transport Stream |`adts_frame`| |`adts_frame` |Audio Data Transport Stream frame |`aac_frame`| |`apev2` |APEv2 metadata tag |`image`| |`ar` |Unix archive |`probe`| |[`asn1_ber`](#asn1_ber) |ASN1 Basic Encoding Rules (also CER and DER) || |`av1_ccr` |AV1 Codec Configuration Record || |`av1_frame` |AV1 frame |`av1_obu`| |`av1_obu` |AV1 Open Bitstream Unit || |`avc_annexb` |H.264/AVC Annex B |`avc_nalu`| |`avc_au` |H.264/AVC Access Unit |`avc_nalu`| |`avc_dcr` |H.264/AVC Decoder Configuration Record |`avc_nalu`| |`avc_nalu` |H.264/AVC Network Access Layer Unit |`avc_sps` `avc_pps` `avc_sei`| |`avc_pps` |H.264/AVC Picture Parameter Set || |`avc_sei` |H.264/AVC Supplemental Enhancement Information || |`avc_sps` |H.264/AVC Sequence Parameter Set || |`avro_ocf` |Avro object container file || |`bencode` |BitTorrent bencoding || |`bsd_loopback_frame` |BSD loopback frame |`ipv4_packet`| |[`bson`](#bson) |Binary JSON || |`bzip2` |bzip2 compression |`probe`| |[`cbor`](#cbor) |Concise Binary Object Representation || |`dns` |DNS packet || |`dns_tcp` |DNS packet (TCP) || |`elf` |Executable and Linkable Format || |`ether8023_frame` |Ethernet 802.3 frame |`ipv4_packet`| |`exif` |Exchangeable Image File Format || |`flac` |Free Lossless Audio Codec file |`flac_metadatablocks` `flac_frame`| |`flac_frame` |FLAC frame || |`flac_metadatablock` |FLAC metadatablock |`flac_streaminfo` `flac_picture` `vorbis_comment`| |`flac_metadatablocks` |FLAC metadatablocks |`flac_metadatablock`| |`flac_picture` |FLAC metadatablock picture |`image`| |`flac_streaminfo` |FLAC streaminfo || |`gif` |Graphics Interchange Format || |`gzip` |gzip compression |`probe`| |`hevc_annexb` |H.265/HEVC Annex B |`hevc_nalu`| |`hevc_au` |H.265/HEVC Access Unit |`hevc_nalu`| |`hevc_dcr` |H.265/HEVC Decoder Configuration Record |`hevc_nalu`| |`hevc_nalu` |H.265/HEVC Network Access Layer Unit || |`icc_profile` |International Color Consortium profile || |`icmp` |Internet Control Message Protocol || |`id3v1` |ID3v1 metadata || |`id3v11` |ID3v1.1 metadata || |`id3v2` |ID3v2 metadata |`image`| |`ipv4_packet` |Internet protocol v4 packet |`udp_datagram` `tcp_segment` `icmp`| |`jpeg` |Joint Photographic Experts Group file |`exif` `icc_profile`| |`json` |JSON || |[`matroska`](#matroska) |Matroska file |`aac_frame` `av1_ccr` `av1_frame` `avc_au` `avc_dcr` `flac_frame` `flac_metadatablocks` `hevc_au` `hevc_dcr` `image` `mp3_frame` `mpeg_asc` `mpeg_pes_packet` `mpeg_spu` `opus_packet` `vorbis_packet` `vp8_frame` `vp9_cfm` `vp9_frame`| |`mp3` |MP3 file |`id3v2` `id3v1` `id3v11` `apev2` `mp3_frame`| |`mp3_frame` |MPEG audio layer 3 frame |`xing`| |[`mp4`](#mp4) |MPEG-4 file and similar |`aac_frame` `av1_ccr` `av1_frame` `flac_frame` `flac_metadatablocks` `id3v2` `image` `jpeg` `mp3_frame` `avc_au` `avc_dcr` `mpeg_es` `hevc_au` `hevc_dcr` `mpeg_pes_packet` `opus_packet` `protobuf_widevine` `pssh_playready` `vorbis_packet` `vp9_frame` `vpx_ccr`| |`mpeg_asc` |MPEG-4 Audio Specific Config || |`mpeg_es` |MPEG Elementary Stream |`mpeg_asc` `vorbis_packet`| |`mpeg_pes` |MPEG Packetized elementary stream |`mpeg_pes_packet` `mpeg_spu`| |`mpeg_pes_packet` |MPEG Packetized elementary stream packet || |`mpeg_spu` |Sub Picture Unit (DVD subtitle) || |`mpeg_ts` |MPEG Transport Stream || |[`msgpack`](#msgpack) |MessagePack || |`ogg` |OGG file |`ogg_page` `vorbis_packet` `opus_packet` `flac_metadatablock` `flac_frame`| |`ogg_page` |OGG page || |`opus_packet` |Opus packet |`vorbis_comment`| |`pcap` |PCAP packet capture |`link_frame` `tcp_stream` `ipv4_packet`| |`pcapng` |PCAPNG packet capture |`link_frame` `tcp_stream` `ipv4_packet`| |`png` |Portable Network Graphics file |`icc_profile` `exif`| |[`protobuf`](#protobuf) |Protobuf || |`protobuf_widevine` |Widevine protobuf |`protobuf`| |`pssh_playready` |PlayReady PSSH || |`raw` |Raw bits || |`sll2_packet` |Linux cooked capture encapsulation v2 |`ether8023_frame`| |`sll_packet` |Linux cooked capture encapsulation |`ether8023_frame`| |`tar` |Tar archive |`probe`| |`tcp_segment` |Transmission control protocol segment || |`tiff` |Tag Image File Format |`icc_profile`| |`udp_datagram` |User datagram protocol |`udp_payload`| |`vorbis_comment` |Vorbis comment |`flac_picture`| |`vorbis_packet` |Vorbis packet |`vorbis_comment`| |`vp8_frame` |VP8 frame || |`vp9_cfm` |VP9 Codec Feature Metadata || |`vp9_frame` |VP9 frame || |`vpx_ccr` |VPX Codec Configuration Record || |`wav` |WAV file |`id3v2` `id3v1` `id3v11`| |`webp` |WebP image |`vp8_frame`| |`xing` |Xing header || |`zip` |ZIP archive |`probe`| |`image` |Group |`gif` `jpeg` `mp4` `png` `tiff` `webp`| |`link_frame` |Group |`bsd_loopback_frame` `ether8023_frame` `sll2_packet` `sll_packet`| |`probe` |Group |`adts` `ar` `avro_ocf` `bzip2` `elf` `flac` `gif` `gzip` `jpeg` `json` `matroska` `mp3` `mp4` `mpeg_ts` `ogg` `pcap` `pcapng` `png` `tar` `tiff` `wav` `webp` `zip`| |`tcp_stream` |Group |`dns`| |`udp_payload` |Group |`dns`| [#]: sh-end ## Format options Currently the only option is `force` and is used to ignore some format assertion errors. It can be used as a decode option or as a CLI `-o` option: ``` fq -d mp4 -o force=true file.mp4 fq -d raw 'mp4({force: true})' file.mp4 ``` ## Format functions [./formats_collect.sh]: sh-start ### asn1_ber Can also decode CER and BER but with no extra validation. Currently does not support specifying a schema. Supports `torepr` but without schema support it's not that useful: ``` fq -d asn1_ber torepr file.ber ``` There is also `frompem` and `topem` to help working with PEM format: ``` fq -d raw 'frompem | asn1_ber | d' cert.pem ``` If the schema is known and not that complicated it can be reproduced: ``` fq -d asn1_ber 'torepr as $r | ["version", "modulus", "private_exponent", "private_exponen", "prime1", "prime2", "exponent1", "exponent2", "coefficient"] | with_entries({key: .value, value: $r[.key]})' pkcs1.der ``` ### becode Supports `torepr`: ``` fq -d bencode torepr file.torrent ``` ### bson Supports `torepr`: ``` fq -d bson torepr file.bson ``` ### cbor Supports `torepr`: ``` fq -d cbor torepr file.cbor fq -d cbor 'torepr.field' file.cbor fq -d cbor 'torepr | .field' file.cbor fq -d cbor 'torepr | grep("abc")' file.cbor ``` ### matroska Supports `matroska_path`: ``` $ fq 'matroska_path(".Segment.Tracks[0]")' file.mkv │00 01 02 03 04 05 06 07 08 09│0123456789│.elements[1].elements[3]{}: 0x122│ 16 54 ae 6b │ .T.k │ id: "Tracks" (0x1654ae6b) (A Top-Level Element of information with many tracks described.) │ │ │ type: "master" (7) 0x122│ 4d bf │ M. │ size: 3519 0x122│ bf│ .│ elements[0:3]: 0x12c│84 cf 8b db a0 ae 01 00 00 00│..........│ 0x136│00 00 00 78 d7 81 01 73 c5 88│...x...s..│ * │until 0xee9.7 (3519) │ │ ``` ``` $ fq 'first(grep_by(.id == "Tracks")) | matroska_path' test.mkv ".Segment.Tracks" ``` ### mp4 Supports `mp4_path`: ``` $ fq 'mp4_path(".moov.trak[1]")' file.mp4 │00 01 02 03 04 05 06 07 08 09│0123456789│.boxes[3].boxes[1]{}: 0x4f6│ 00 00 02│ ...│ size: 573 0x500│3d │= │ 0x500│ 74 72 61 6b │ trak │ type: "trak" (Container for an individual track or stream) 0x500│ 00 00 00 5c 74│ ...\t│ boxes[0:3]: 0x50a│6b 68 64 00 00 00 03 00 00 00│khd.......│ 0x514│00 00 00 00 00 00 00 00 01 00│..........│ * │until 0x739.7 (565) │ │ ``` ``` $ fq 'first(grep_by(.type == "trak")) | mp4_path' file.mp4 ".moov.trak" ``` ### msgpack Supports `torepr`: ``` fq -d msgpack torepr file.msgpack ``` ### protobuf `protobuf` decoder can be used to decode sub messages: ``` fq -d protobuf '.fields[6].wire_value | protobuf | d' ``` [#]: sh-end ## Dependency graph ![alt text](formats.svg "Format diagram")