1
1
mirror of https://github.com/wader/fq.git synced 2024-12-24 13:52:02 +03:00
jq for binary formats
Go to file
2021-09-19 02:17:31 +02:00
.github/workflows github: Install expect to cli test 2021-09-18 18:11:51 +02:00
.vscode cli: Add -M -C support and default to color if tty 2021-09-19 01:19:48 +02:00
dev Rename and move cli test to pkg/cli 2021-09-18 18:23:28 +02:00
doc cli: Add -M -C support and default to color if tty 2021-09-19 01:19:48 +02:00
format decode: Add UTF8Fn functions and trim some null terminated strings 2021-09-17 22:13:35 +02:00
internal fqtest: Add env support and isterminal/width/height support 2021-09-19 02:17:31 +02:00
pkg fqtest: Add env support and isterminal/width/height support 2021-09-19 02:17:31 +02:00
.golangci.yml funcs: Remove unsued string function 2021-09-12 13:08:53 +02:00
.goreleaser.yml Init 2021-09-12 13:08:42 +02:00
Bumpfile bump: Add action and cleanup names 2021-09-12 13:08:52 +02:00
Dockerfile docker: Make image build again and install expect 2021-09-18 18:12:36 +02:00
go.mod gojq: Update fq fork, support JQValue alt // 2021-09-16 00:46:25 +02:00
go.sum gojq: Update fq fork, support JQValue alt // 2021-09-16 00:46:25 +02:00
LICENSE Init 2021-09-12 13:08:42 +02:00
main.go cli: Make profile build optional and move it to cli 2021-09-12 13:08:53 +02:00
Makefile make: Move doc generate to helper script 2021-09-18 18:24:12 +02:00
README.md doc: Some fixes 2021-09-19 00:51:15 +02:00

fq

Tool, language and format decoders for exploring binary data.

# Overview of mp3 file 
$ fq . file.mp3 
     |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f|0123456789abcdef|.: {} file.mp3 (mp3)
0x000|49 44 33 04 00 00 00 00 15 39 54 53 53 45 00 00|ID3......9TSSE..|  headers: [1]
*    |until 0xac2.7 (2755)                           |                |
0xac0|         ff fb 40 c0 00 00 00 00 00 00 00 00 00|   ..@..........|  frames: [3]
0xad0|00 00 00 00 00 00 00 00 49 6e 66 6f 00 00 00 0f|........Info....|
*    |until 0xd19.7 (end) (599)                      |                |
     |                                               |                |  footers: [0]
 
# Show ID3v2 tag in mp3 file 
$ fq '.headers[0]' file.mp3 
     |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f|0123456789abcdef|.headers[0]: {} (id3v2)
0x000|49 44 33                                       |ID3             |  magic: "ID3" (Correct)
0x000|         04                                    |   .            |  version: 4
0x000|            00                                 |    .           |  revision: 0
0x000|               00                              |     .          |  flags: {}
0x000|                  00 00 15 39                  |      ...9      |  size: 2745
0x000|                              54 53 53 45 00 00|          TSSE..|  frames: [2]
0x010|00 0f 00 00 03 4c 61 76 66 35 38 2e 37 36 2e 31|.....Lavf58.76.1|
*    |until 0xab8.7 (2735)                           |                |
0xab0|                           00 00 00 00 00 00 00|         .......|  padding: Correct (none) (zero padding)
0xac0|00 00 00                                       |...             |
 
# Resolution of ID3v2 cover art 
$ fq '.headers[0].frames[] | select(.id == "APIC").picture.chunks[] | select(.type == "IHDR") | {width, height}' file.mp3 
{
  "height": 240,
  "width": 320
}
 
# Extract image file 
$ fq '.headers[].frames[] | select(.id == "APIC")?.picture | tobits' file.mp3 > file.png 
$ file file.png 
file.png: PNG image data, 320 x 240, 8-bit/color RGB, non-interlaced
 

Goals

  • Make structured binary data accessible.
  • Nested formats and bit-oriented decoding.
  • Quick CLI tool that mimics jq as much as possible.
  • Do bits and bytes transformation and conversions.
  • Programmer's calculator.

NOTE: fq is early in development and many things are missing, broken or do not make sense. That also means there is a great opportunity to help out.

Install

Download archive from releases page for your platform, unarchive and move the executable to PATH.

Homebrew

# install latest release
brew install wader/tap/fq

Build from source

Make sure you have go 1.17 or later and then do:

# build and install latest master
go install github.com/wader/fq@latest

and the binary should end up at $GOPATH/bin/fq.

Usage

Basic usage is:

Usage: fq [OPTIONS] [--] [EXPR] [FILE...]

For more usage details see usage.md.

Supported formats

aac_frame, adts, adts_frame, apev2, av1_ccr, av1_frame, av1_obu, avc_annexb, avc_au, avc_dcr, avc_nalu, avc_pps, avc_sei, avc_sps, bzip2, dns, elf, exif, flac, flac_frame, flac_metadatablocks, flac_picture, gif, gzip, hevc_annexb, hevc_au, hevc_dcr, hevc_nalu, icc_profile, id3v1, id3v11, id3v2, jpeg, json, matroska, mp3, mp3_frame, mp4, mpeg_asc, mpeg_es, mpeg_pes, mpeg_pes_packet, mpeg_spu, mpeg_ts, ogg, ogg_page, opus_packet, png, protobuf, protobuf_widevine, pssh_playready, raw, tar, tiff, vorbis_comment, vorbis_packet, vp8_frame, vp9_cfm, vp9_frame, vpx_ccr, wav, webp, xing

For more format details see usage.md.

TODO and ideas

See TODO.md

Development

See dev.md

This project would not be possible without itchyny's jq implementation gojq. Also want to thank HexFiend for inspiration and ideas and stedolan for inventing the jq language.

Similar projects: