wader/fq
1
1
Fork 0
mirror of https://github.com/wader/fq.git synced 2024-12-23 05:13:30 +03:00
Code Issues Projects Releases Wiki Activity

doc: Improve readme a bit and add torepr example

Browse Source
This commit is contained in:
Mattias Wadman 2022-01-15 19:26:45 +01:00
parent 63160bf52e
commit 8e9700d0bd
2 changed files with 30 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

24
README.md
View File

@ -1,27 +1,29 @@
# fq # fq
Tool, language and decoders for inspecting binary data. Tool, language and decoders for working with binary data.
![fq demo](doc/demo.svg) ![fq demo](doc/demo.svg)
In most cases fq works the same way as jq but instead of reading JSON it reads binary data. In most cases fq works the same way as jq but instead of reading JSON it reads binary data.
The result is a JSON compatible structures where each value has a bit range, symbolic The result is a JSON compatible structure where values have a bit range, can have symbolic
interpretations and know how to be presented in a useful way. values and know how to be presented in a useful ways.
You can pronounce the name as you wish, i pronounce jq /‘dʒei’kju:/ so I usually prefer /‘ef’kju:/. It was initially developed to debug, inspect and query media files but has since been extended
to handle a variety of binary formats.
**NOTE:** fq is early in development and many things are missing, broken or do not make sense. You can pronounce the name as you wish, I pronounce jq /‘dʒei’kju:/ so I usually pronounce fq /‘ef’kju:/.
That also means there is a great opportunity to help out!
## Goals **NOTE:** fq is still early in development so some things are broken or do not make sense.
That also means that there is a great opportunity to help out!
### Goals
- Make binary formats accessible and queryable. - Make binary formats accessible and queryable.
- Nested formats and bit-oriented decoding. - Nested formats and bit-oriented decoding.
- Quick and comfortable CLI tool. - Quick and comfortable CLI tool.
- Bit and byte transformations and conversions. - Bits and bytes transformations.
- Programmer's calculator.
## Hopes ### Hopes
- Make it useful enough that people want to help improve it. - Make it useful enough that people want to help improve it.
- Inspire people to create similar tools. - Inspire people to create similar tools.
@ -29,7 +31,7 @@ That also means there is a great opportunity to help out!
Basic usage is `fq . file`. Basic usage is `fq . file`.
For details see [usage.md](doc/usage.md) For more details see [usage.md](doc/usage.md)
## Install ## Install

18
doc/usage.md
View File

@ -103,16 +103,19 @@ Use Ctrl-D to exit and Ctrl-C to interrupt current evaluation.
## Example usages ## Example usages
#### Second mp3 frame header as JSON #### Second mp3 frame header as JSON
```sh ```sh
fq '.frames[1].header | tovalue' file.mp3 fq '.frames[1].header | tovalue' file.mp3
``` ```
#### Byte start position for the first 10 mp3 frames in an array #### Byte start position for the first 10 mp3 frames in an array
```sh ```sh
fq '.frames[0:10] | map(tobytesrange.start)' file.mp3 fq '.frames[0:10] | map(tobytesrange.start)' file.mp3
``` ```
#### Decode at range #### Decode at range
```sh ```sh
# decode byte range 100 to end # decode byte range 100 to end
fq -d raw 'tobytes[100:] | mp3_frame | d' file.mp3 fq -d raw 'tobytes[100:] | mp3_frame | d' file.mp3
@ -128,6 +131,7 @@ decoded value for `a.mp4` and `b.mp4` filtered thru `f`.
```sh ```sh
fq -n 'def f: .. | select(format=="avc_sps"); diff(input|f; input|f)' a.mp4 b.mp4 fq -n 'def f: .. | select(format=="avc_sps"); diff(input|f; input|f)' a.mp4 b.mp4
``` ```
#### Extract first JPEG found in file #### Extract first JPEG found in file
Recursively look for first value that is a `jpeg` decode value root. Use `tobytes` to get bytes buffer for value. Redirect bytes to a file. Recursively look for first value that is a `jpeg` decode value root. Use `tobytes` to get bytes buffer for value. Redirect bytes to a file.
@ -152,7 +156,19 @@ Use `grep` to recursively find strings matching a regexp.
fq '.tcp_connections | grep("GET /.* HTTP/1.?")' file.pcap fq '.tcp_connections | grep("GET /.* HTTP/1.?")' file.pcap
``` ```
### #### Use representation of a format
Some formats like `msgpack`, `bson` etc are used to represent some data structure. In those cases the `torepr`
function can be used to get the representation.
```sh
# whole represented value
fq -d msgpack torepr file.msgpack
# value of the key "field" from the represented value
fq -d msgpack `torepr.field` file.msgpack
# query or transform represented value
fq -d msgpack 'torepr | ...' file.msgpack
```
#### Widest PNG in a directory #### Widest PNG in a directory
```sh ```sh