wader/fq
1
1
Fork 0
mirror of https://github.com/wader/fq.git synced 2024-10-27 12:19:52 +03:00
Code Issues Projects Releases Wiki Activity

Merge branch 'postgres' into postgres_wal_sync_with_postgres

Browse Source
This commit is contained in:
Pavel Safonov 2022-10-12 10:43:59 +03:00
commit dd9c7f0420
289 changed files with 14273 additions and 6041 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitattributes vendored
View File

@ -3,3 +3,4 @@
*.json eol=lf
*.jq eol=lf
*.xml eol=lf
*.md eol=lf

6
.github/workflows/ci.yml vendored
View File

@ -7,7 +7,7 @@ on:
pull_request:
env:
GOLANGCILINT_VERSION: 1.49.0
GOLANGCILINT_VERSION: 1.50.0
jobs:
lint:
@ -15,7 +15,7 @@ jobs:
steps:
- uses: actions/setup-go@v3
with:
go-version: 1.19.1
go-version: 1.19.2
- uses: actions/checkout@v3
- uses: golangci/golangci-lint-action@v3
with:
@ -47,7 +47,7 @@ jobs:
- uses: actions/checkout@v3
- uses: actions/setup-go@v3
with:
go-version: 1.19.1
go-version: 1.19.2
- name: Test
env:
GOARCH: ${{ matrix.goarch }}

2
.github/workflows/release.yml vendored
View File

@ -15,7 +15,7 @@ jobs:
- uses: actions/checkout@v3
- uses: actions/setup-go@v3
with:
go-version: 1.19.1
go-version: 1.19.2
- uses: goreleaser/goreleaser-action@v2
with:
distribution: goreleaser

4
.vscode/settings.json vendored
View File

@ -154,14 +154,18 @@
"tfhd",
"tfra",
"tmpl",
"toactual",
"toarray",
"toboolean",
"tobytes",
"tobytesrange",
"todescription",
"toimage",
"tojson",
"tojvalue",
"topath",
"torepr",
"tosym",
"tovalue",
"toxml",
"toxmlentities",

175
CHANGES.md
View File

@ -1,3 +1,178 @@
# 0.0.10
## Changes
- Add `bplist` Apple Binary Property List decoder. Thanks David McDonald @dgmcdona #427
- Add `markdown` decoder. #422
- Fix panic when interrupting (ctrl-c) JSON output (`fq tovalue file ` etc), #440
- Fix issue using `debug` (and some other native go iterator functions) inside `path(...)`, which is used by assign (`... = ...`) expressions etc. #439
- Fix issue and also make `toactual` and `tosym` work more similar to `tovalue`. #432
- Fix issue with unknown fields (gaps found after decoding) where one continuous gap could end up split into two of more unknown fields. #431
- More format documentation and also nicer help output. Also now all documentation is in markdown format. #430 #422
```
# or help(matroska) in the REPL
$ fq -h matroska
matroska: Matroska file decoder
Decode examples
===============
# Decode file as matroska
$ fq -d matroska . file
# Decode value as matroska
... | matroska
Lookup element using path
=========================
$ fq 'matroska_path(".Segment.Tracks[0)")' file.mkv
Get path to element
===================
$ fq 'grep_by(.id == "Tracks") | matroska_path' file.mkv
References
==========
- https://tools.ietf.org/html/draft-ietf-cellar-ebml-00
- https://matroska.org/technical/specs/index.html
- https://www.matroska.org/technical/basics.html
- https://www.matroska.org/technical/codec_specs.html
- https://wiki.xiph.org/MatroskaOpus
```
## Decoder changes
- `ar` Allow empty integer strings. For example owner id can be an empty string. #428
- `bitcoin_blkdat` Assert that there is a header. As the format is part of the probe group this speeds up probing. #402
- `bplist` Add Apple Binary Property List decoder.
```sh
$ fq '.objects.entries[0] | .key, .value' Info.plist
│00 01 02 03 04 05 06 07 08 09│0123456789│.objects.entries[0].key{}:
0x32│ 5c │ \ │ type: "ascii_string" (5) (ASCII encoded string)
0x32│ 5c │ \ │ size_bits: 12
│ │ │ size: 12
0x32│ 43 46 42 75│ CFBu│ value: "CFBundleName"
0x3c│6e 64 6c 65 4e 61 6d 65 │ndleName │
│00 01 02 03 04 05 06 07 08 09│0123456789│.objects.entries[0].value{}:
0x1ea│ 5f │ _ │ type: "ascii_string" (5) (ASCII encoded string)
0x1ea│ 5f │ _ │ size_bits: 15
0x1ea│ 10 │ . │ large_size_marker: 1 (valid)
0x1ea│ 10 │ . │ exponent: 0
0x1ea│ 18 │ . │ size_bigint: 24
│ │ │ size: 24
0x1ea│ 41 70 70 6c│ Appl│ value: "AppleProResCodecEmbedded"
0x1f4│65 50 72 6f 52 65 73 43 6f 64│eProResCod│
0x1fe│65 63 45 6d 62 65 64 64 65 64│ecEmbedded│
```
- Supports `torepr`
```sh
$ fq torepr.CFBundleName Info.plist
"AppleProResCodecEmbedded"
```
- `elf`
- More robust decoding when string tables are missing. #417<br>
```sh
# extract entry opcodes and disassemble with ndisasm
$ fq -n '"f0VMRgIBAQAAAAAAAAAAAAIAPgABAAAAeABAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAEAAOAABAAAAAAAAAAEAAAAFAAAAAAAAAAAAAAAAAEAAAAAAAAAAQAAAAAAAAAAAAAEAAAAAAAAAAQAAAAAAIAAAAAAAsDxmvwYADwU=" | frombase64 | . as $b | elf | $b[.header.entry-.program_headers[0].vaddr:]' \
| ndisasm -b 64 -
00000000 B03C mov al,0x3c
00000002 66BF0600 mov di,0x6
00000006 0F05 syscall
```
- Now decodes program header notes. #421
- `markdown` Add decoder. Is used in fq to render CLI help. #422
```sh
# array with all level 1 and 2 headers
$ fq -d markdown '[.. | select(.type=="heading" and .level<=2)?.children[0]]' README.md
[
"fq",
"Usage",
"Presentations",
"Install",
"TODO and ideas",
"Development and adding a new decoder",
"Thanks and related projects",
"License"
]
```
- `matroska` Add support for sample lacing. Used by FLAC samples etc. #404
- `mp4` More codec names and also use official names from mp4ra.org. #424<br>
```sh
# show details of first two track in file
$ fq -o line_bytes=10 '.tracks[0,1]' big_buck_bunny.mp4
│00 01 02 03 04 05 06 07 08 09│0123456789│.tracks[0]{}: track
0x00910a│20 68 10 01 a0 40 0e 20 8c 1b│ h...@. ..│ samples[0:1295]:
0x009114│c2 2b 99 09 84 42 60 a8 c4 60│.+...B`..`│
* │until 0x541697.7 (5473678) │ │
│ │ │ id: 1
│ │ │ data_format: "mp4a" (MPEG-4 Audio)
│00 01 02 03 04 05 06 07 08 09│0123456789│.tracks[1]{}: track
0x00a5e6│ 00│ .│ samples[0:1440]:
0x00a5f0│00 00 0c 06 00 07 8b 71 b0 00│.......q..│
0x00a5fa│00 03 00 40 80 00 00 00 15 06│...@......│
* │until 0x540959.7 (5464939) │ │
│ │ │ id: 2
│ │ │ data_format: "avc1" (Advanced Video Coding)
```
- `html` Handle leading doc type and processing directives. #414
## Changelog
* a77cec92 Added documentation and tests, fixed bad date parsing
* d784db69 Adds support for Apple Binary Plist, version 00
* 5711f290 Code fixes from PR, still need to add tests and testdata
* 6b04f2de Documentation cleanup
* bcccde23 Fixes and embeds documentation
* ebae938d Fixes bug in integer parsing
* 368d183b Size check on nBits to save memory
* 84ca1010 Update docker-golang from 1.19.0 to 1.19.1
* c47c3866 Update github-go-version from 1.19.0, 1.19.0, 1.19.0 to 1.19.1
* 816169b6 Update github-golangci-lint to 1.50.0 from 1.49.0
* 21f2980e Update make-golangci-lint to 1.50.0 from 1.49.0
* 5f619940 adds function for decoding fixed sized arrays
* f08f44f1 ar: Integer strings might be empty
* 004406de bitcoin_blkdat,bitcoin_block: Make sure there is a header if blkdat
* 421b2b30 bplist: Fix unknown field for singletons and add torepr tests
* 16b01211 bplist: Make torepr convert to values
* fe64530e csv: Add tsv and header example
* cb3dc802 decode,tar: Add scalar description and Try* helpers
* a6429ffe decode: Remove RangeSorted flag as we can decide on array/struct instead
* a468684a deps: Manual update ones not using bump
* a7a101ca doc,help: Nicer format help and move help tests into each format
* 725ab1b1 doc,html,xml: Add more documentation and examples
* abd19ed8 doc: Fix format sections a bit
* 0fdc03a4 doc: Fix some incorrect example prompts
* 5382d46a elf: Basic program header notes decoding
* 12105d8c elf: Treat missing string tables as empty to be more robust
* 3deceeeb fixes from PR comments
* 226a9a3e generics: Use more from x/exp
* 404b1704 gojq: Update fq fork
* 376f0ebb gojq: Update rebased fq fork
* 87b2c6c1 help,doc: Use markdown for format documentation again
* 8016352b html: Handle html with leading doctype etc
* 768df301 interp,decode: For struct use map to lookup field
* c4219d69 interp: Fix interrupt panic for cli eval
* 00ee10a1 interp: Make to{actual,sym} behave similar to tovalue
* 00a50662 markdown: Add decoder
* 7749e1b5 matroska: Add proper lacing support
* 20a15372 mp4: Fix data_format typo
* 2655ba09 mp4: More codec names (from mp4ra.org)
* 7cd43b49 perfomance: increase performance by map usage
* 6a6fec54 range,decode: Use own range sort impl to speed up a bit
* 0f35fe48 ranges,decode: Correctly skip empty ranges when adding unknown fields
* ea81efec readline: Update rebased fq fork
* 369f4016 removed unneccessary type conversions
* 3198602d removed unused return type
* 7d865343 sortex: Package with type safe sort helpers
* 808202fa test: Skip go test with -race by default
* 12836abe updates fqtest
* 1e47f4f2 updates tests post integer-bug fix
* 3d8ea1de updates torepr for data type
* 1385b5d0 wasm: Add some documentation
* d6316d5c wav: Decode smpl chunk
# 0.0.9
## Changes

2
Dockerfile
View File

@ -1,5 +1,5 @@
# bump: docker-golang /FROM golang:([\d.]+)/ docker:golang|^1
FROM golang:1.19.1-bullseye AS base
FROM golang:1.19.2-bullseye AS base
# expect is used to test cli
RUN \

2
Makefile
View File

@ -69,7 +69,7 @@ gogenerate:
lint:
# bump: make-golangci-lint /golangci-lint@v([\d.]+)/ git:https://github.com/golangci/golangci-lint.git|^1
# bump: make-golangci-lint link "Release notes" https://github.com/golangci/golangci-lint/releases/tag/v$LATEST
go run github.com/golangci/golangci-lint/cmd/golangci-lint@v1.49.0 run
go run github.com/golangci/golangci-lint/cmd/golangci-lint@v1.50.0 run
.PHONY: depgraph.svg
depgraph.svg:

12
README.md
View File

@ -48,9 +48,10 @@ avc_sps,
[avro_ocf](doc/formats.md#avro_ocf),
[bencode](doc/formats.md#bencode),
bitcoin_blkdat,
bitcoin_block,
[bitcoin_block](doc/formats.md#bitcoin_block),
bitcoin_script,
bitcoin_transaction,
[bplist](doc/formats.md#bplist),
bsd_loopback_frame,
[bson](doc/formats.md#bson),
bzip2,
@ -91,6 +92,7 @@ json,
jsonl,
[macho](doc/formats.md#macho),
macho_fat,
[markdown](doc/formats.md#markdown),
[matroska](doc/formats.md#matroska),
[mp3](doc/formats.md#mp3),
mp3_frame,
@ -105,8 +107,12 @@ mpeg_ts,
ogg,
ogg_page,
opus_packet,
pcap,
[pcap](doc/formats.md#pcap),
pcapng,
[pg_btree](doc/formats.md#pg_btree),
[pg_control](doc/formats.md#pg_control),
[pg_heap](doc/formats.md#pg_heap),
[pg_wal](doc/formats.md#pg_wal),
png,
prores_frame,
[protobuf](doc/formats.md#protobuf),
@ -127,7 +133,7 @@ vp8_frame,
vp9_cfm,
vp9_frame,
vpx_ccr,
wasm,
[wasm](doc/formats.md#wasm),
wav,
webp,
xing,

33
doc/formats.jq
View File

@ -2,14 +2,13 @@
def code: "`\(.)`";
def nbsp: gsub(" "; "&nbsp;");
def has_section($f; $fhelp): $fhelp.notes or $fhelp.examples or $fhelp.links or $f.decode_in_arg;
def has_section($f; $fhelp): $fhelp.notes or $fhelp.examples or $f.decode_in_arg or ((_registry.files[][] | select(.name=="\($f.name).md").data) // false);
def formats_list:
[ formats
| to_entries[] as {$key, $value}
| (_format_func($key; "_help")? // {}) as $fhelp
| if has_section($value; $fhelp) then "[\($key)](doc/formats.md#\($key))"
else $key
[ formats[] as $f
| ({} | _help_format_enrich("fq"; $f; false)) as $fhelp
| if has_section($f; $fhelp) then "[\($f.name)](doc/formats.md#\($f.name))"
else $f.name
end
] | join(",\n");
@ -71,13 +70,14 @@ def formats_table:
def formats_sections:
( formats[] as $f
| (_format_func($f.name; "_help")? // {} | _help_format_enrich("fq"; $f; false)) as $fhelp
| ((_registry.files[][] | select(.name=="\($f.name).md").data) // false) as $doc
| ({} | _help_format_enrich("fq"; $f; false)) as $fhelp
| select(has_section($f; $fhelp))
| "### \($f.name)"
| "## \($f.name)"
, ""
, ($fhelp.notes | if . then ., "" else empty end)
, if $f.decode_in_arg then
( "#### Options"
( "### Options"
, ""
, ( [ { name: "Name"
, default: "Default"
@ -113,7 +113,7 @@ def formats_sections:
else empty
end
, if $fhelp.examples then
( "#### Examples"
( "### Examples"
, ""
, ( $fhelp.examples[]
| "\(.comment)"
@ -134,16 +134,5 @@ def formats_sections:
)
else empty
end
, if $fhelp.links then
( "#### References and links"
, ""
, ( $fhelp.links[]
| if .title then "- [\(.title)](\(.url))"
else "- \(.url)"
end
)
, ""
)
else empty
end
, ($doc // empty)
);

776
doc/formats.md
View File

File diff suppressed because it is too large Load Diff

1030
doc/formats.svg
View File

File diff suppressed because it is too large Load Diff
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 134 KiB

After

Width:  |  Height:  |  Size: 135 KiB

283
doc/postgres.md
View File

@ -1,283 +0,0 @@
## PostgreSQL formats
### Supported PostgreSQL data types (`fq -d pg_heap`):
- `pg_heap` - heap, table, relation
- `pg_control` - control file data
- `pg_wal` - wal, write ahead log (not implemented yet)
- `pg_btree` - btree index (not implemented yet)
### Supported PostgreSQL flavours (`fq -o flavour=postgres14`):
1. Standart (vanilla) PostgreSQL
- `postgres11`
- `postgres12`
- `postgres13`
- `postgres14`
2. Postgres Pro Enterprise
- `pgproee10`
- `pgproee11`
- `pgproee12`
- `pgproee13`
- `pgproee14`
3. Postgres Professional Standard
- `pgpro11`
- `pgpro12`
- `pgpro13`
- `pgpro14`
### Supported OS
Postgres for x64 Linux deb, rpm based OS.
### How to run?
Need to specify format, flavour for file and expression:
```
fq -d pg_control -o flavour=postgres14 d pg_control
fq -d pg_heap -o flavour=postgres14 ".[]" 16397
```
### pg_control file format
Source code:
- [pg_control.h](https://github.com/postgres/postgres/blob/master/src/include/catalog/pg_control.h)
To see content of pg_control run:
```
$ fq -d pg_control -o flavour=postgres14 d pg_control
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f|0123456789abcdef|.{}: pg_control (pg_control)
0x000|68 e7 dd 05 b3 88 b3 61 |h......a | system_identifier: 7040120944989169512
0x000| 14 05 00 00 | .... | pg_control_version: 1300
0x000| 2d e9 0b 0c| -...| catalog_version_no: 202107181
0x010|06 00 00 00 |.... | state: "DB_IN_PRODUCTION" (6)
0x010| 46 a1 c2 62 00 00 00 00| F..b....| time: "Mon, 04 Jul 2022 08:13:58 UTC" (1656922438)
0x020|c0 88 d6 9a 00 00 00 00 |........ | checkPoint: "0/9AD688C0" (2597750976)
| | | checkPointCopy{}:
0x020| c0 88 d6 9a 00 00 00 00| ........| redo: "0/9AD688C0" (2597750976)
0x030|01 00 00 00 |.... | ThisTimeLineID: 1
0x030| 01 00 00 00 | .... | PrevTimeLineID: 1
0x030| 01 | . | fullPageWrites: 1
0x040|4b ab 1c 00 00 00 00 00 |K....... | nextXid: 1878859
0x040| dd 81 00 00 | .... | nextOid: 33245
0x040| 01 00 00 00| ....| nextMulti: 1
0x050|00 00 00 00 |.... | nextMultiOffset: 0
0x050| d6 02 00 00 | .... | oldestXid: 726
0x050| 01 00 00 00 | .... | oldestXidDB: 1
0x050| 01 00 00 00| ....| oldestMulti: 1
0x060|01 00 00 00 |.... | oldestMultiDB: 1
0x060| 46 a1 c2 62 00 00 00 00| F..b....| time: "Mon, 04 Jul 2022 08:13:58 UTC" (1656922438)
0x070|00 00 00 00 |.... | oldestCommitTsXid: 0
0x070| 00 00 00 00 | .... | newestCommitTsXid: 0
0x070| 00 00 00 00 | .... | oldestActiveXid: 0
0x080|e8 03 00 00 00 00 00 00 |........ | unloggedLSN: "0/3E8" (1000)
0x080| 00 00 00 00 00 00 00 00| ........| minRecoveryPoint: "0/0" (0)
0x090|00 00 00 00 |.... | minRecoveryPointTLI: 0
0x090| 00 00 00 00 00 00 00 00| ........| backupStartPoint: "0/0" (0)
0x0a0|00 00 00 00 00 00 00 00 |........ | backupEndPoint: "0/0" (0)
0x0a0| 00 | . | backupEndRequired: 0
0x0a0| 01 00 00 00| ....| wal_level: "WAL_LEVEL_REPLICA" (1)
0x0b0|00 |. | wal_log_hints: 0
0x0b0| 64 00 00 00 | d... | MaxConnections: 100
0x0b0| 08 00 00 00 | .... | max_worker_processes: 8
0x0b0| 0a 00 00 00| ....| max_wal_senders: 10
0x0c0|00 00 00 00 |.... | max_prepared_xacts: 0
0x0c0| 40 00 00 00 | @... | max_locks_per_xact: 64
0x0c0| 00 | . | track_commit_timestamp: 0
0x0c0| 08 00 00 00| ....| maxAlign: 8
0x0d0|00 00 00 00 87 d6 32 41 |......2A | floatFormat: 1.234567e+06
0x0d0| 00 20 00 00 | . .. | blcksz: 8192
0x0d0| 00 00 02 00| ....| relseg_size: 131072
0x0e0|00 20 00 00 |. .. | xlog_blcksz: 8192
0x0e0| 00 00 00 01 | .... | xlog_seg_size: 16777216
0x0e0| 40 00 00 00 | @... | nameDataLen: 64
0x0e0| 20 00 00 00| ...| indexMaxKeys: 32
0x0f0|cc 07 00 00 |.... | toast_max_chunk_size: 1996
0x0f0| 00 08 00 00 | .... | loblksize: 2048
0x0f0| 01 | . | float8ByVal: 1
0x0f0| 00 00 00 00| ....| data_checksum_version: 0
0x100|00 45 fd 64 7e d4 d3 53 82 75 0a b7 d6 be c1 9a|.E.d~..S.u......| mock_authentication_nonce: "0045fd647ed4d35382750ab7d6bec19a77af72bae00f728..." (raw bits)
0x110|77 af 72 ba e0 0f 72 80 4a 57 43 fb 76 c8 98 8c|w.r...r.JWC.v...|
0x120|4b 76 27 eb |Kv'. | crc: 3945231947
```
Specific fields can be got by request:
```
$ fq -d pg_control -o flavour=postgres14 ".state, .checkPointCopy.redo, .wal_level" pg_control
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f|0123456789abcdef|
0x10|06 00 00 00 |.... |.state: "DB_IN_PRODUCTION" (6)
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f|0123456789abcdef|
0x20| c0 88 d6 9a 00 00 00 00| ........|.checkPointCopy.redo: "0/9AD688C0" (2597750976)
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f|0123456789abcdef|
0xa0| 01 00 00 00| ....|.wal_level: "WAL_LEVEL_REPLICA" (1)
```
### Heap data format
Heap page structure:
- [https://www.postgresql.org/docs/current/storage-page-layout.html](https://www.postgresql.org/docs/current/storage-page-layout.html)
- [https://postgrespro.ru/docs/postgresql/14/storage-page-layout?lang=en](https://postgrespro.ru/docs/postgresql/14/storage-page-layout?lang=en)
Heap consists of pages. You can see page content:
```
$ fq -d pg_heap -o flavour=postgres14 ".[0]" 16397
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13|0123456789abcdef0123|.[0]{}: HeapPage
0x0000|00 00 00 00 f0 aa 72 01 00 00 04 00 0c 01 80 01 00 20 04 20|......r.......... . | PageHeaderData{}:
* |until 0x10b.7 (268) | |
0x0104| 00 00 00 00 00 00 00 00 00 00 00 00| ............| FreeSpace: "00000000000000000000000000000000000000000000000..." (raw bits)
0x0118|00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00|....................|
* |until 0x17f.7 (116) | |
0x017c| e2 02 00 00 00 00 00 00 50 04 00 00 00 00 00 00| ........P.......| Tuples[0:61]:
0x0190|3d 00 04 00 02 09 18 00 3d 00 00 00 01 00 00 00 00 00 00 00|=.......=...........|
* |until 0x1ff8.7 (7801)
```
To get PageHeaderData you can use this request:
```
$ fq -d pg_heap -o flavour=postgres14 ".[0].PageHeaderData" 16397
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13|0123456789abcdef0123|.[0].PageHeaderData{}:
0x000|00 00 00 00 f0 aa 72 01 |......r. | pd_lsn{}:
0x000| 00 00 | .. | pd_checksum: 0
0x000| 04 00 | .. | pd_flags: 4
0x000| 0c 01 | .. | pd_lower: 268
0x000| 80 01 | .. | pd_upper: 384
0x000| 00 20 | . | pd_special: 8192
0x000| 04 20| . | pd_pagesize_version: 8196
0x014|00 00 00 00 |.... | pd_prune_xid: 0
0x014| 80 9f f2 00 00 9f f2 00 80 9e f2 00 00 9e f2 00| ................| pd_linp[0:61]:
0x028|80 9d f2 00 00 9d f2 00 80 9c f2 00 00 9c f2 00 80 9b f2 00|....................|
* |until 0x10b.7 (244)
```
To get first and last item pointers on first page:
```
$ fq -d pg_heap -o flavour=postgres14 ".[0].PageHeaderData.pd_linp[0, -1]" 16397
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13|0123456789abcdef0123|.[0].PageHeaderData.pd_linp[0]{}: ItemIdData
0x14| 80 9f f2 00 | .... | lp_off: 8064
0x14| 80 9f f2 00 | .... | lp_flags: "LP_NORMAL" (1)
0x14| 80 9f f2 00 | .... | lp_len: 121
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13|0123456789abcdef0123|.[0].PageHeaderData.pd_linp[60]{}: ItemIdData
0x104| 80 81 f2 00 | .... | lp_off: 384
0x104| 80 81 f2 00 | .... | lp_flags: "LP_NORMAL" (1)
0x104| 80 81 f2 00 | .... | lp_len: 121
```
First and last tuple on first page:
```
$ fq -d pg_heap -o flavour=postgres14 ".[0].Tuples[0, -1]" 16397
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13|0123456789abcdef0123|.[0].Tuples[0]{}: HeapTupleHeaderData
0x1f7c| e2 02 00 00 00 00 00 00 50 04 00 00 | ........P... | t_choice{}:
0x1f7c| 00 00 00 00| ....| t_ctid{}:
0x1f90|01 00 |.. |
0x1f90| 04 00 | .. | t_infomask2: 4
0x1f90| 04 00 | .. | Infomask2{}:
0x1f90| 02 09 | .. | t_infomask: 2306
0x1f90| 02 09 | .. | Infomask{}:
0x1f90| 18 | . | t_hoff: 24
0x1f90| 01 00 00 00 01 00 00 00 00 00 00 00| ............| t_bits: "010000000100000000000000ab202020202020202020202..." (raw bits)
0x1fa4|ab 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20|. |
* |until 0x1ff8.7 (97) | |
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13|0123456789abcdef0123|.[0].Tuples[60]{}: HeapTupleHeaderData
0x17c| e2 02 00 00 00 00 00 00 50 04 00 00 | ........P... | t_choice{}:
0x17c| 00 00 00 00| ....| t_ctid{}:
0x190|3d 00 |=. |
0x190| 04 00 | .. | t_infomask2: 4
0x190| 04 00 | .. | Infomask2{}:
0x190| 02 09 | .. | t_infomask: 2306
0x190| 02 09 | .. | Infomask{}:
0x190| 18 | . | t_hoff: 24
0x190| 3d 00 00 00 01 00 00 00 00 00 00 00| =...........| t_bits: "3d0000000100000000000000ab202020202020202020202..." (raw bits)
0x1a4|ab 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20|. |
* |until 0x1f8.7 (97)
```
Tuple contains a lot of field:
```
$ fq -d pg_heap -o flavour=postgres14 ".[0].Tuples[0] | d" 16397
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13|0123456789abcdef0123|.[0].Tuples[0]{}: HeapTupleHeaderData
| | | t_choice{}:
| | | t_heap{}:
0x1f7c| e2 02 00 00 | .... | t_xmin: 738
0x1f7c| 00 00 00 00 | .... | t_xmax: 0
| | | t_field3{}:
0x1f7c| 50 04 00 00 | P... | t_cid: 1104
0x1f7c| 50 04 00 00 | P... | t_xvac: 1104
| | | t_datum{}:
0x1f7c| e2 02 00 00 | .... | datum_len_: 738
0x1f7c| 00 00 00 00 | .... | datum_typmod: 0
0x1f7c| 50 04 00 00 | P... | datum_typeid: 1104
| | | t_ctid{}:
0x1f7c| 00 00 00 00| ....| ip_blkid: 0
0x1f90|01 00 |.. | ip_posid: 1
0x1f90| 04 00 | .. | t_infomask2: 4
| | | Infomask2{}:
0x1f90| 04 00 | .. | HEAP_KEYS_UPDATED: 0
0x1f90| 04 00 | .. | HEAP_HOT_UPDATED: 0
0x1f90| 04 00 | .. | HEAP_ONLY_TUPLE: 0
0x1f90| 02 09 | .. | t_infomask: 2306
| | | Infomask{}:
0x1f90| 02 09 | .. | HEAP_HASNULL: 0
0x1f90| 02 09 | .. | HEAP_HASVARWIDTH: 1
0x1f90| 02 09 | .. | HEAP_HASEXTERNAL: 0
0x1f90| 02 09 | .. | HEAP_HASOID_OLD: 0
0x1f90| 02 09 | .. | HEAP_XMAX_KEYSHR_LOCK: 0
0x1f90| 02 09 | .. | HEAP_COMBOCID: 0
0x1f90| 02 09 | .. | HEAP_XMAX_EXCL_LOCK: 0
0x1f90| 02 09 | .. | HEAP_XMAX_LOCK_ONLY: 0
0x1f90| 02 09 | .. | HEAP_XMAX_SHR_LOCK: 0
0x1f90| 02 09 | .. | HEAP_LOCK_MASK: 0
0x1f90| 02 09 | .. | HEAP_XMIN_COMMITTED: 1
0x1f90| 02 09 | .. | HEAP_XMIN_INVALID: 0
0x1f90| 02 09 | .. | HEAP_XMIN_FROZEN: 0
0x1f90| 02 09 | .. | HEAP_XMAX_COMMITTED: 0
0x1f90| 02 09 | .. | HEAP_XMAX_INVALID: 1
0x1f90| 02 09 | .. | HEAP_XMAX_IS_MULTI: 0
0x1f90| 02 09 | .. | HEAP_UPDATED: 0
0x1f90| 02 09 | .. | HEAP_MOVED_OFF: 0
0x1f90| 02 09 | .. | HEAP_MOVED_IN: 0
0x1f90| 02 09 | .. | HEAP_MOVED: 0
0x1f90| 18 | . | t_hoff: 24
0x1f90| 01 00 00 00 01 00 00 00 00 00 00 00| ............| t_bits: "010000000100000000000000ab202020202020202020202..." (raw bits)
0x1fa4|ab 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20|. |
* |until 0x1ff8.7 (97)
```
Some of tuple fields are C unions:
```
$ time fq -d pg_heap -o flavour=postgres14 ".[0].Tuples[0].t_choice | d" 16397
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13|0123456789abcdef0123|.[0].Tuples[0].t_choice{}:
| | | t_heap{}:
0x1f7c| e2 02 00 00 | .... | t_xmin: 738
0x1f7c| 00 00 00 00 | .... | t_xmax: 0
| | | t_field3{}:
0x1f7c| 50 04 00 00 | P... | t_cid: 1104
0x1f7c| 50 04 00 00 | P... | t_xvac: 1104
| | | t_datum{}:
0x1f7c| e2 02 00 00 | .... | datum_len_: 738
0x1f7c| 00 00 00 00 | .... | datum_typmod: 0
0x1f7c| 50 04 00 00 | P... | datum_typeid: 1104
```
Some tuple fields are flags:
```
$ time fq -d pg_heap -o flavour=postgres14 ".[0].Tuples[0] | .t_infomask, .Infomask" 16397
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13|0123456789abcdef0123|
0x1f90| 02 09 | .. |.[0].Tuples[0].t_infomask: 2306
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13|0123456789abcdef0123|.[0].Tuples[0].Infomask{}:
0x1f90| 02 09 | .. | HEAP_HASNULL: 0
0x1f90| 02 09 | .. | HEAP_HASVARWIDTH: 1
0x1f90| 02 09 | .. | HEAP_HASEXTERNAL: 0
0x1f90| 02 09 | .. | HEAP_HASOID_OLD: 0
0x1f90| 02 09 | .. | HEAP_XMAX_KEYSHR_LOCK: 0
0x1f90| 02 09 | .. | HEAP_COMBOCID: 0
0x1f90| 02 09 | .. | HEAP_XMAX_EXCL_LOCK: 0
0x1f90| 02 09 | .. | HEAP_XMAX_LOCK_ONLY: 0
0x1f90| 02 09 | .. | HEAP_XMAX_SHR_LOCK: 0
0x1f90| 02 09 | .. | HEAP_LOCK_MASK: 0
0x1f90| 02 09 | .. | HEAP_XMIN_COMMITTED: 1
0x1f90| 02 09 | .. | HEAP_XMIN_INVALID: 0
0x1f90| 02 09 | .. | HEAP_XMIN_FROZEN: 0
0x1f90| 02 09 | .. | HEAP_XMAX_COMMITTED: 0
0x1f90| 02 09 | .. | HEAP_XMAX_INVALID: 1
0x1f90| 02 09 | .. | HEAP_XMAX_IS_MULTI: 0
0x1f90| 02 09 | .. | HEAP_UPDATED: 0
0x1f90| 02 09 | .. | HEAP_MOVED_OFF: 0
0x1f90| 02 09 | .. | HEAP_MOVED_IN: 0
0x1f90| 02 09 | .. | HEAP_MOVED: 0
```

4
doc/usage.md
View File

@ -372,8 +372,8 @@ unary uses input and if more than one argument all as arguments ignoring the inp
- `parents` output parents of value
- `topath` path of value. Use `path_to_expr` to get a string representation.
- `tovalue`, `tovalue($opts)` symbolic value if available otherwise actual value
- `toactual` actual value (decoded etc)
- `tosym` symbolic value (mapped etc)
- `toactual`, `toactual($opts)` actual value (usually the decoded value)
- `tosym`, `tosym($opts)` symbolic value (mapped etc)
- `todescription` description of value
- `torepr` convert decode value into what it reptresents. For example convert msgpack decode value
into a value representing its JSON representation.

110
format/all/all.fqtest
View File

@ -4,6 +4,7 @@ $ fq -n _registry.groups.probe
"ar",
"avro_ocf",
"bitcoin_blkdat",
"bplist",
"bzip2",
"elf",
"flac",
@ -32,3 +33,112 @@ $ fq -n _registry.groups.probe
"xml",
"yaml"
]
$ fq --help formats
aac_frame Advanced Audio Coding frame
adts Audio Data Transport Stream
adts_frame Audio Data Transport Stream frame
amf0 Action Message Format 0
apev2 APEv2 metadata tag
ar Unix archive
asn1_ber ASN1 BER (basic encoding rules, also CER and DER)
av1_ccr AV1 Codec Configuration Record
av1_frame AV1 frame
av1_obu AV1 Open Bitstream Unit
avc_annexb H.264/AVC Annex B
avc_au H.264/AVC Access Unit
avc_dcr H.264/AVC Decoder Configuration Record
avc_nalu H.264/AVC Network Access Layer Unit
avc_pps H.264/AVC Picture Parameter Set
avc_sei H.264/AVC Supplemental Enhancement Information
avc_sps H.264/AVC Sequence Parameter Set
avro_ocf Avro object container file
bencode BitTorrent bencoding
bitcoin_blkdat Bitcoin blk.dat
bitcoin_block Bitcoin block
bitcoin_script Bitcoin script
bitcoin_transaction Bitcoin transaction
bplist Apple Binary Property List
bsd_loopback_frame BSD loopback frame
bson Binary JSON
bzip2 bzip2 compression
cbor Concise Binary Object Representation
csv Comma separated values
dns DNS packet
dns_tcp DNS packet (TCP)
elf Executable and Linkable Format
ether8023_frame Ethernet 802.3 frame
exif Exchangeable Image File Format
fairplay_spc FairPlay Server Playback Context
flac Free Lossless Audio Codec file
flac_frame FLAC frame
flac_metadatablock FLAC metadatablock
flac_metadatablocks FLAC metadatablocks
flac_picture FLAC metadatablock picture
flac_streaminfo FLAC streaminfo
gif Graphics Interchange Format
gzip gzip compression
hevc_annexb H.265/HEVC Annex B
hevc_au H.265/HEVC Access Unit
hevc_dcr H.265/HEVC Decoder Configuration Record
hevc_nalu H.265/HEVC Network Access Layer Unit
hevc_pps H.265/HEVC Picture Parameter Set
hevc_sps H.265/HEVC Sequence Parameter Set
hevc_vps H.265/HEVC Video Parameter Set
html HyperText Markup Language
icc_profile International Color Consortium profile
icmp Internet Control Message Protocol
icmpv6 Internet Control Message Protocol v6
id3v1 ID3v1 metadata
id3v11 ID3v1.1 metadata
id3v2 ID3v2 metadata
ipv4_packet Internet protocol v4 packet
ipv6_packet Internet protocol v6 packet
jpeg Joint Photographic Experts Group file
json JavaScript Object Notation
jsonl JavaScript Object Notation Lines
macho Mach-O macOS executable
macho_fat Fat Mach-O macOS executable (multi-architecture)
markdown Markdown
matroska Matroska file
mp3 MP3 file
mp3_frame MPEG audio layer 3 frame
mp4 ISOBMFF, QuickTime and similar
mpeg_asc MPEG-4 Audio Specific Config
mpeg_es MPEG Elementary Stream
mpeg_pes MPEG Packetized elementary stream
mpeg_pes_packet MPEG Packetized elementary stream packet
mpeg_spu Sub Picture Unit (DVD subtitle)
mpeg_ts MPEG Transport Stream
msgpack MessagePack
ogg OGG file
ogg_page OGG page
opus_packet Opus packet
pcap PCAP packet capture
pcapng PCAPNG packet capture
png Portable Network Graphics file
prores_frame Apple ProRes frame
protobuf Protobuf
protobuf_widevine Widevine protobuf
pssh_playready PlayReady PSSH
raw Raw bits
rtmp Real-Time Messaging Protocol
sll2_packet Linux cooked capture encapsulation v2
sll_packet Linux cooked capture encapsulation
tar Tar archive
tcp_segment Transmission control protocol segment
tiff Tag Image File Format
toml Tom's Obvious, Minimal Language
udp_datagram User datagram protocol
vorbis_comment Vorbis comment
vorbis_packet Vorbis packet
vp8_frame VP8 frame
vp9_cfm VP9 Codec Feature Metadata
vp9_frame VP9 frame
vpx_ccr VPX Codec Configuration Record
wasm WebAssembly Binary Format
wav WAV file
webp WebP image
xing Xing header
xml Extensible Markup Language
yaml YAML Ain't Markup Language
zip ZIP archive

2
format/all/all.go
View File

@ -11,6 +11,7 @@ import (
_ "github.com/wader/fq/format/avro"
_ "github.com/wader/fq/format/bencode"
_ "github.com/wader/fq/format/bitcoin"
_ "github.com/wader/fq/format/bplist"
_ "github.com/wader/fq/format/bson"
_ "github.com/wader/fq/format/bzip2"
_ "github.com/wader/fq/format/cbor"
@ -28,6 +29,7 @@ import (
_ "github.com/wader/fq/format/jpeg"
_ "github.com/wader/fq/format/json"
_ "github.com/wader/fq/format/macho"
_ "github.com/wader/fq/format/markdown"
_ "github.com/wader/fq/format/math"
_ "github.com/wader/fq/format/matroska"
_ "github.com/wader/fq/format/mp3"

908
format/all/help.fqtest
View File

@ -1,908 +0,0 @@
$ fq -n '_registry.formats | keys[] | "help(\(.))", ([_help("fq"; .)] | join("\n") | split("\n")[] | ("out \(.)" | println))'
"help(aac_frame)"
out aac_frame: Advanced Audio Coding frame decoder
out Options:
out object_type=1 Audio object type
out Examples:
out # Decode file as aac_frame
out $ fq -d aac_frame . file
out # Decode value as aac_frame
out ... | aac_frame
out # Decode file using aac_frame options
out $ fq -d aac_frame -o object_type=1 . file
out # Decode value as aac_frame
out ... | aac_frame({object_type:1})
"help(adts)"
out adts: Audio Data Transport Stream decoder
out Examples:
out # Decode file as adts
out $ fq -d adts . file
out # Decode value as adts
out ... | adts
"help(adts_frame)"
out adts_frame: Audio Data Transport Stream frame decoder
out Examples:
out # Decode file as adts_frame
out $ fq -d adts_frame . file
out # Decode value as adts_frame
out ... | adts_frame
"help(amf0)"
out amf0: Action Message Format 0 decoder
out Examples:
out # Decode file as amf0
out $ fq -d amf0 . file
out # Decode value as amf0
out ... | amf0
"help(apev2)"
out apev2: APEv2 metadata tag decoder
out Examples:
out # Decode file as apev2
out $ fq -d apev2 . file
out # Decode value as apev2
out ... | apev2
"help(ar)"
out ar: Unix archive decoder
out Examples:
out # Decode file as ar
out $ fq -d ar . file
out # Decode value as ar
out ... | ar
"help(asn1_ber)"
out asn1_ber: ASN1 BER (basic encoding rules, also CER and DER) decoder
out Supports decoding BER, CER and DER (X.690).
out
out - Currently no extra validation is done for CER and DER.
out - Does not support specifying a schema.
out - Supports torepr but without schema all sequences and sets will be arrays.
out Examples:
out # frompem` and `topem can be used to work with PEM format
out $ fq -d raw 'frompem | asn1_ber | d' cert.pem
out # Can be used to decode nested parts
out $ fq -d asn1_ber '.constructed[1].value | asn1_ber' file.ber
out # If schema is known and not complicated it can be reproduced
out $ fq -d asn1_ber 'torepr as $r | ["version", "modulus", "private_exponent", "private_exponen", "prime1", "prime2", "exponent1", "exponent2", "coefficient"] | with_entries({key: .value, value: $r[.key]})' pkcs1.der
out # Decode file as asn1_ber
out $ fq -d asn1_ber . file
out # Decode value as asn1_ber
out ... | asn1_ber
out # Supports torepr
out $ fq -d asn1_ber torepr file
out # Supports torepr
out ... | asn1_ber | torepr
out References and links
out https://www.itu.int/ITU-T/studygroups/com10/languages/X.690_1297.pdf
out https://en.wikipedia.org/wiki/X.690
out https://letsencrypt.org/docs/a-warm-welcome-to-asn1-and-der/
out https://lapo.it/asn1js/
"help(av1_ccr)"
out av1_ccr: AV1 Codec Configuration Record decoder
out Examples:
out # Decode file as av1_ccr
out $ fq -d av1_ccr . file
out # Decode value as av1_ccr
out ... | av1_ccr
"help(av1_frame)"
out av1_frame: AV1 frame decoder
out Examples:
out # Decode file as av1_frame
out $ fq -d av1_frame . file
out # Decode value as av1_frame
out ... | av1_frame
"help(av1_obu)"
out av1_obu: AV1 Open Bitstream Unit decoder
out Examples:
out # Decode file as av1_obu
out $ fq -d av1_obu . file
out # Decode value as av1_obu
out ... | av1_obu
"help(avc_annexb)"
out avc_annexb: H.264/AVC Annex B decoder
out Examples:
out # Decode file as avc_annexb
out $ fq -d avc_annexb . file
out # Decode value as avc_annexb
out ... | avc_annexb
"help(avc_au)"
out avc_au: H.264/AVC Access Unit decoder
out Options:
out length_size=4 Length value size
out Examples:
out # Decode file as avc_au
out $ fq -d avc_au . file
out # Decode value as avc_au
out ... | avc_au
out # Decode file using avc_au options
out $ fq -d avc_au -o length_size=4 . file
out # Decode value as avc_au
out ... | avc_au({length_size:4})
"help(avc_dcr)"
out avc_dcr: H.264/AVC Decoder Configuration Record decoder
out Examples:
out # Decode file as avc_dcr
out $ fq -d avc_dcr . file
out # Decode value as avc_dcr
out ... | avc_dcr
"help(avc_nalu)"
out avc_nalu: H.264/AVC Network Access Layer Unit decoder
out Examples:
out # Decode file as avc_nalu
out $ fq -d avc_nalu . file
out # Decode value as avc_nalu
out ... | avc_nalu
"help(avc_pps)"
out avc_pps: H.264/AVC Picture Parameter Set decoder
out Examples:
out # Decode file as avc_pps
out $ fq -d avc_pps . file
out # Decode value as avc_pps
out ... | avc_pps
"help(avc_sei)"
out avc_sei: H.264/AVC Supplemental Enhancement Information decoder
out Examples:
out # Decode file as avc_sei
out $ fq -d avc_sei . file
out # Decode value as avc_sei
out ... | avc_sei
"help(avc_sps)"
out avc_sps: H.264/AVC Sequence Parameter Set decoder
out Examples:
out # Decode file as avc_sps
out $ fq -d avc_sps . file
out # Decode value as avc_sps
out ... | avc_sps
"help(avro_ocf)"
out avro_ocf: Avro object container file decoder
out Supports reading Avro Object Container Format (OCF) files based on the 1.11.0 specification.
out
out Capable of handling null, deflate, and snappy codecs for data compression.
out
out Limitations:
out - Schema does not support self-referential types, only built-in types.
out - Decimal logical types are not supported for decoding, will just be treated as their primitive type
out Examples:
out # Decode file as avro_ocf
out $ fq -d avro_ocf . file
out # Decode value as avro_ocf
out ... | avro_ocf
out References and links
out https://avro.apache.org/docs/current/spec.html#Object+Container+Files
"help(bencode)"
out bencode: BitTorrent bencoding decoder
out Examples:
out # Decode file as bencode
out $ fq -d bencode . file
out # Decode value as bencode
out ... | bencode
out # Supports torepr
out $ fq -d bencode torepr file
out # Supports torepr
out ... | bencode | torepr
out References and links
out https://wiki.theory.org/BitTorrentSpecification#Bencoding
"help(bitcoin_blkdat)"
out bitcoin_blkdat: Bitcoin blk.dat decoder
out Examples:
out # Decode file as bitcoin_blkdat
out $ fq -d bitcoin_blkdat . file
out # Decode value as bitcoin_blkdat
out ... | bitcoin_blkdat
"help(bitcoin_block)"
out bitcoin_block: Bitcoin block decoder
out Options:
out has_header=false Has blkdat header
out Examples:
out # Decode file as bitcoin_block
out $ fq -d bitcoin_block . file
out # Decode value as bitcoin_block
out ... | bitcoin_block
out # Decode file using bitcoin_block options
out $ fq -d bitcoin_block -o has_header=false . file
out # Decode value as bitcoin_block
out ... | bitcoin_block({has_header:false})
"help(bitcoin_script)"
out bitcoin_script: Bitcoin script decoder
out Examples:
out # Decode file as bitcoin_script
out $ fq -d bitcoin_script . file
out # Decode value as bitcoin_script
out ... | bitcoin_script
"help(bitcoin_transaction)"
out bitcoin_transaction: Bitcoin transaction decoder
out Examples:
out # Decode file as bitcoin_transaction
out $ fq -d bitcoin_transaction . file
out # Decode value as bitcoin_transaction
out ... | bitcoin_transaction
"help(bsd_loopback_frame)"
out bsd_loopback_frame: BSD loopback frame decoder
out Examples:
out # Decode file as bsd_loopback_frame
out $ fq -d bsd_loopback_frame . file
out # Decode value as bsd_loopback_frame
out ... | bsd_loopback_frame
"help(bson)"
out bson: Binary JSON decoder
out Examples:
out # BSON as JSON
out $ fq -d bson torepr file
out # Decode file as bson
out $ fq -d bson . file
out # Decode value as bson
out ... | bson
out # Supports torepr
out $ fq -d bson torepr file
out # Supports torepr
out ... | bson | torepr
out References and links
out https://wiki.theory.org/BitTorrentSpecification#Bencoding
"help(bzip2)"
out bzip2: bzip2 compression decoder
out Examples:
out # Decode file as bzip2
out $ fq -d bzip2 . file
out # Decode value as bzip2
out ... | bzip2
"help(cbor)"
out cbor: Concise Binary Object Representation decoder
out Examples:
out # Decode file as cbor
out $ fq -d cbor . file
out # Decode value as cbor
out ... | cbor
out # Supports torepr
out $ fq -d cbor torepr file
out # Supports torepr
out ... | cbor | torepr
out References and links
out https://en.wikipedia.org/wiki/CBOR
out https://www.rfc-editor.org/rfc/rfc8949.html
"help(csv)"
out csv: Comma separated values decoder
out Options:
out comma=, Separator character
out comment=# Comment line character
out Examples:
out # Decode file as csv
out $ fq -d csv . file
out # Decode value as csv
out ... | csv
out # Decode file using csv options
out $ fq -d csv -o comma="," -o comment="#" . file
out # Decode value as csv
out ... | csv({comma:",",comment:"#"})
"help(dns)"
out dns: DNS packet decoder
out Examples:
out # Decode file as dns
out $ fq -d dns . file
out # Decode value as dns
out ... | dns
"help(dns_tcp)"
out dns_tcp: DNS packet (TCP) decoder
out Examples:
out # Decode file as dns_tcp
out $ fq -d dns_tcp . file
out # Decode value as dns_tcp
out ... | dns_tcp
"help(elf)"
out elf: Executable and Linkable Format decoder
out Examples:
out # Decode file as elf
out $ fq -d elf . file
out # Decode value as elf
out ... | elf
"help(ether8023_frame)"
out ether8023_frame: Ethernet 802.3 frame decoder
out Examples:
out # Decode file as ether8023_frame
out $ fq -d ether8023_frame . file
out # Decode value as ether8023_frame
out ... | ether8023_frame
"help(exif)"
out exif: Exchangeable Image File Format decoder
out Examples:
out # Decode file as exif
out $ fq -d exif . file
out # Decode value as exif
out ... | exif
"help(fairplay_spc)"
out fairplay_spc: FairPlay Server Playback Context decoder
out Examples:
out # Decode file as fairplay_spc
out $ fq -d fairplay_spc . file
out # Decode value as fairplay_spc
out ... | fairplay_spc
"help(flac)"
out flac: Free Lossless Audio Codec file decoder
out Examples:
out # Decode file as flac
out $ fq -d flac . file
out # Decode value as flac
out ... | flac
"help(flac_frame)"
out flac_frame: FLAC frame decoder
out Options:
out bits_per_sample=16 Bits per sample
out Examples:
out # Decode file as flac_frame
out $ fq -d flac_frame . file
out # Decode value as flac_frame
out ... | flac_frame
out # Decode file using flac_frame options
out $ fq -d flac_frame -o bits_per_sample=16 . file
out # Decode value as flac_frame
out ... | flac_frame({bits_per_sample:16})
"help(flac_metadatablock)"
out flac_metadatablock: FLAC metadatablock decoder
out Examples:
out # Decode file as flac_metadatablock
out $ fq -d flac_metadatablock . file
out # Decode value as flac_metadatablock
out ... | flac_metadatablock
"help(flac_metadatablocks)"
out flac_metadatablocks: FLAC metadatablocks decoder
out Examples:
out # Decode file as flac_metadatablocks
out $ fq -d flac_metadatablocks . file
out # Decode value as flac_metadatablocks
out ... | flac_metadatablocks
"help(flac_picture)"
out flac_picture: FLAC metadatablock picture decoder
out Examples:
out # Decode file as flac_picture
out $ fq -d flac_picture . file
out # Decode value as flac_picture
out ... | flac_picture
"help(flac_streaminfo)"
out flac_streaminfo: FLAC streaminfo decoder
out Examples:
out # Decode file as flac_streaminfo
out $ fq -d flac_streaminfo . file
out # Decode value as flac_streaminfo
out ... | flac_streaminfo
"help(gif)"
out gif: Graphics Interchange Format decoder
out Examples:
out # Decode file as gif
out $ fq -d gif . file
out # Decode value as gif
out ... | gif
"help(gzip)"
out gzip: gzip compression decoder
out Examples:
out # Decode file as gzip
out $ fq -d gzip . file
out # Decode value as gzip
out ... | gzip
"help(hevc_annexb)"
out hevc_annexb: H.265/HEVC Annex B decoder
out Examples:
out # Decode file as hevc_annexb
out $ fq -d hevc_annexb . file
out # Decode value as hevc_annexb
out ... | hevc_annexb
"help(hevc_au)"
out hevc_au: H.265/HEVC Access Unit decoder
out Options:
out length_size=4 Length value size
out Examples:
out # Decode file as hevc_au
out $ fq -d hevc_au . file
out # Decode value as hevc_au
out ... | hevc_au
out # Decode file using hevc_au options
out $ fq -d hevc_au -o length_size=4 . file
out # Decode value as hevc_au
out ... | hevc_au({length_size:4})
"help(hevc_dcr)"
out hevc_dcr: H.265/HEVC Decoder Configuration Record decoder
out Examples:
out # Decode file as hevc_dcr
out $ fq -d hevc_dcr . file
out # Decode value as hevc_dcr
out ... | hevc_dcr
"help(hevc_nalu)"
out hevc_nalu: H.265/HEVC Network Access Layer Unit decoder
out Examples:
out # Decode file as hevc_nalu
out $ fq -d hevc_nalu . file
out # Decode value as hevc_nalu
out ... | hevc_nalu
"help(hevc_pps)"
out hevc_pps: H.265/HEVC Picture Parameter Set decoder
out Examples:
out # Decode file as hevc_pps
out $ fq -d hevc_pps . file
out # Decode value as hevc_pps
out ... | hevc_pps
"help(hevc_sps)"
out hevc_sps: H.265/HEVC Sequence Parameter Set decoder
out Examples:
out # Decode file as hevc_sps
out $ fq -d hevc_sps . file
out # Decode value as hevc_sps
out ... | hevc_sps
"help(hevc_vps)"
out hevc_vps: H.265/HEVC Video Parameter Set decoder
out Examples:
out # Decode file as hevc_vps
out $ fq -d hevc_vps . file
out # Decode value as hevc_vps
out ... | hevc_vps
"help(html)"
out html: HyperText Markup Language decoder
out Options:
out array=false Decode as nested arrays
out attribute_prefix=@ Prefix for attribute keys
out seq=false Use seq attribute to preserve element order
out Examples:
out # Decode file as html
out $ fq -d html . file
out # Decode value as html
out ... | html
out # Decode file using html options
out $ fq -d html -o array=false -o attribute_prefix="@" -o seq=false . file
out # Decode value as html
out ... | html({array:false,attribute_prefix:"@",seq:false})
"help(icc_profile)"
out icc_profile: International Color Consortium profile decoder
out Examples:
out # Decode file as icc_profile
out $ fq -d icc_profile . file
out # Decode value as icc_profile
out ... | icc_profile
"help(icmp)"
out icmp: Internet Control Message Protocol decoder
out Examples:
out # Decode file as icmp
out $ fq -d icmp . file
out # Decode value as icmp
out ... | icmp
"help(icmpv6)"
out icmpv6: Internet Control Message Protocol v6 decoder
out Examples:
out # Decode file as icmpv6
out $ fq -d icmpv6 . file
out # Decode value as icmpv6
out ... | icmpv6
"help(id3v1)"
out id3v1: ID3v1 metadata decoder
out Examples:
out # Decode file as id3v1
out $ fq -d id3v1 . file
out # Decode value as id3v1
out ... | id3v1
"help(id3v11)"
out id3v11: ID3v1.1 metadata decoder
out Examples:
out # Decode file as id3v11
out $ fq -d id3v11 . file
out # Decode value as id3v11
out ... | id3v11
"help(id3v2)"
out id3v2: ID3v2 metadata decoder
out Examples:
out # Decode file as id3v2
out $ fq -d id3v2 . file
out # Decode value as id3v2
out ... | id3v2
"help(ipv4_packet)"
out ipv4_packet: Internet protocol v4 packet decoder
out Examples:
out # Decode file as ipv4_packet
out $ fq -d ipv4_packet . file
out # Decode value as ipv4_packet
out ... | ipv4_packet
"help(ipv6_packet)"
out ipv6_packet: Internet protocol v6 packet decoder
out Examples:
out # Decode file as ipv6_packet
out $ fq -d ipv6_packet . file
out # Decode value as ipv6_packet
out ... | ipv6_packet
"help(jpeg)"
out jpeg: Joint Photographic Experts Group file decoder
out Examples:
out # Decode file as jpeg
out $ fq -d jpeg . file
out # Decode value as jpeg
out ... | jpeg
"help(json)"
out json: JavaScript Object Notation decoder
out Examples:
out # Decode file as json
out $ fq -d json . file
out # Decode value as json
out ... | json
"help(jsonl)"
out jsonl: JavaScript Object Notation Lines decoder
out Examples:
out # Decode file as jsonl
out $ fq -d jsonl . file
out # Decode value as jsonl
out ... | jsonl
"help(macho)"
out macho: Mach-O macOS executable decoder
out Supports decoding vanilla and FAT Mach-O binaries.
out Examples:
out # Select 64bit load segments
out $ fq '.load_commands[] | select(.cmd=="segment_64")' file
out # Decode file as macho
out $ fq -d macho . file
out # Decode value as macho
out ... | macho
out References and links
out https://github.com/aidansteele/osx-abi-macho-file-format-reference
"help(macho_fat)"
out macho_fat: Fat Mach-O macOS executable (multi-architecture) decoder
out Examples:
out # Decode file as macho_fat
out $ fq -d macho_fat . file
out # Decode value as macho_fat
out ... | macho_fat
"help(matroska)"
out matroska: Matroska file decoder
out Examples:
out # Lookup element decode value using matroska_path
out ... | matroska_path(".Segment.Tracks[0)"
out # Return matroska_path string for a box decode value
out ... | grep_by(.id == "Tracks") | matroska_path
out # Decode file as matroska
out $ fq -d matroska . file
out # Decode value as matroska
out ... | matroska
out References and links
out https://tools.ietf.org/html/draft-ietf-cellar-ebml-00
out https://matroska.org/technical/specs/index.html
out https://www.matroska.org/technical/basics.html
out https://www.matroska.org/technical/codec_specs.html
out https://wiki.xiph.org/MatroskaOpus
"help(mp3)"
out mp3: MP3 file decoder
out Options:
out max_sync_seek=32768 Max byte distance to next sync
out max_unique_header_configs=5 Max number of unique frame header configs allowed
out Examples:
out # Decode file as mp3
out $ fq -d mp3 . file
out # Decode value as mp3
out ... | mp3
out # Decode file using mp3 options
out $ fq -d mp3 -o max_sync_seek=32768 -o max_unique_header_configs=5 . file
out # Decode value as mp3
out ... | mp3({max_sync_seek:32768,max_unique_header_configs:5})
"help(mp3_frame)"
out mp3_frame: MPEG audio layer 3 frame decoder
out Examples:
out # Decode file as mp3_frame
out $ fq -d mp3_frame . file
out # Decode value as mp3_frame
out ... | mp3_frame
"help(mp4)"
out mp4: ISOBMFF MPEG-4 part 12 and similar decoder
out Support mp4_path
out Options:
out allow_truncated=false Allow box to be truncated
out decode_samples=true Decode supported media samples
out Examples:
out # Lookup box decode value using mp4_path
out ... | mp4_path(".moov.trak[1]")
out # Return mp4_path string for a box decode value
out ... | grep_by(.type == "trak") | mp4_path
out # Decode file as mp4
out $ fq -d mp4 . file
out # Decode value as mp4
out ... | mp4
out # Decode file using mp4 options
out $ fq -d mp4 -o allow_truncated=false -o decode_samples=true . file
out # Decode value as mp4
out ... | mp4({allow_truncated:false,decode_samples:true})
out References and links
out ISO/IEC base media file format (MPEG-4 Part 12) https://en.wikipedia.org/wiki/ISO/IEC_base_media_file_format
out Quicktime file format https://developer.apple.com/standards/qtff-2001.pdf
"help(mpeg_asc)"
out mpeg_asc: MPEG-4 Audio Specific Config decoder
out Examples:
out # Decode file as mpeg_asc
out $ fq -d mpeg_asc . file
out # Decode value as mpeg_asc
out ... | mpeg_asc
"help(mpeg_es)"
out mpeg_es: MPEG Elementary Stream decoder
out Examples:
out # Decode file as mpeg_es
out $ fq -d mpeg_es . file
out # Decode value as mpeg_es
out ... | mpeg_es
"help(mpeg_pes)"
out mpeg_pes: MPEG Packetized elementary stream decoder
out Examples:
out # Decode file as mpeg_pes
out $ fq -d mpeg_pes . file
out # Decode value as mpeg_pes
out ... | mpeg_pes
"help(mpeg_pes_packet)"
out mpeg_pes_packet: MPEG Packetized elementary stream packet decoder
out Examples:
out # Decode file as mpeg_pes_packet
out $ fq -d mpeg_pes_packet . file
out # Decode value as mpeg_pes_packet
out ... | mpeg_pes_packet
"help(mpeg_spu)"
out mpeg_spu: Sub Picture Unit (DVD subtitle) decoder
out Examples:
out # Decode file as mpeg_spu
out $ fq -d mpeg_spu . file
out # Decode value as mpeg_spu
out ... | mpeg_spu
"help(mpeg_ts)"
out mpeg_ts: MPEG Transport Stream decoder
out Examples:
out # Decode file as mpeg_ts
out $ fq -d mpeg_ts . file
out # Decode value as mpeg_ts
out ... | mpeg_ts
"help(msgpack)"
out msgpack: MessagePack decoder
out Examples:
out # Decode file as msgpack
out $ fq -d msgpack . file
out # Decode value as msgpack
out ... | msgpack
out # Supports torepr
out $ fq -d msgpack torepr file
out # Supports torepr
out ... | msgpack | torepr
out References and links
out https://github.com/msgpack/msgpack/blob/master/spec.md
"help(ogg)"
out ogg: OGG file decoder
out Examples:
out # Decode file as ogg
out $ fq -d ogg . file
out # Decode value as ogg
out ... | ogg
"help(ogg_page)"
out ogg_page: OGG page decoder
out Examples:
out # Decode file as ogg_page
out $ fq -d ogg_page . file
out # Decode value as ogg_page
out ... | ogg_page
"help(opus_packet)"
out opus_packet: Opus packet decoder
out Examples:
out # Decode file as opus_packet
out $ fq -d opus_packet . file
out # Decode value as opus_packet
out ... | opus_packet
"help(pcap)"
out pcap: PCAP packet capture decoder
out Examples:
out # Decode file as pcap
out $ fq -d pcap . file
out # Decode value as pcap
out ... | pcap
"help(pcapng)"
out pcapng: PCAPNG packet capture decoder
out Examples:
out # Decode file as pcapng
out $ fq -d pcapng . file
out # Decode value as pcapng
out ... | pcapng
"help(png)"
out png: Portable Network Graphics file decoder
out Examples:
out # Decode file as png
out $ fq -d png . file
out # Decode value as png
out ... | png
"help(prores_frame)"
out prores_frame: Apple ProRes frame decoder
out Examples:
out # Decode file as prores_frame
out $ fq -d prores_frame . file
out # Decode value as prores_frame
out ... | prores_frame
"help(protobuf)"
out protobuf: Protobuf decoder
out Examples:
out # Can be used to decode sub messages
out $ fq -d protobuf '.fields[6].wire_value | protobuf | d'
out # Decode file as protobuf
out $ fq -d protobuf . file
out # Decode value as protobuf
out ... | protobuf
out References and links
out https://developers.google.com/protocol-buffers/docs/encoding
"help(protobuf_widevine)"
out protobuf_widevine: Widevine protobuf decoder
out Examples:
out # Decode file as protobuf_widevine
out $ fq -d protobuf_widevine . file
out # Decode value as protobuf_widevine
out ... | protobuf_widevine
"help(pssh_playready)"
out pssh_playready: PlayReady PSSH decoder
out Examples:
out # Decode file as pssh_playready
out $ fq -d pssh_playready . file
out # Decode value as pssh_playready
out ... | pssh_playready
"help(raw)"
out raw: Raw bits decoder
out Examples:
out # Decode file as raw
out $ fq -d raw . file
out # Decode value as raw
out ... | raw
"help(rtmp)"
out rtmp: Real-Time Messaging Protocol decoder
out Current only supports plain RTMP (not RTMPT or encrypted variants etc) with AMF0 (not AMF3).
out Examples:
out # Decode file as rtmp
out $ fq -d rtmp . file
out # Decode value as rtmp
out ... | rtmp
out References and links
out https://rtmp.veriskope.com/docs/spec/
out https://rtmp.veriskope.com/pdf/video_file_format_spec_v10.pdf
"help(sll2_packet)"
out sll2_packet: Linux cooked capture encapsulation v2 decoder
out Examples:
out # Decode file as sll2_packet
out $ fq -d sll2_packet . file
out # Decode value as sll2_packet
out ... | sll2_packet
"help(sll_packet)"
out sll_packet: Linux cooked capture encapsulation decoder
out Examples:
out # Decode file as sll_packet
out $ fq -d sll_packet . file
out # Decode value as sll_packet
out ... | sll_packet
"help(tar)"
out tar: Tar archive decoder
out Examples:
out # Decode file as tar
out $ fq -d tar . file
out # Decode value as tar
out ... | tar
"help(tcp_segment)"
out tcp_segment: Transmission control protocol segment decoder
out Examples:
out # Decode file as tcp_segment
out $ fq -d tcp_segment . file
out # Decode value as tcp_segment
out ... | tcp_segment
"help(tiff)"
out tiff: Tag Image File Format decoder
out Examples:
out # Decode file as tiff
out $ fq -d tiff . file
out # Decode value as tiff
out ... | tiff
"help(toml)"
out toml: Tom's Obvious, Minimal Language decoder
out Examples:
out # Decode file as toml
out $ fq -d toml . file
out # Decode value as toml
out ... | toml
"help(udp_datagram)"
out udp_datagram: User datagram protocol decoder
out Examples:
out # Decode file as udp_datagram
out $ fq -d udp_datagram . file
out # Decode value as udp_datagram
out ... | udp_datagram
"help(vorbis_comment)"
out vorbis_comment: Vorbis comment decoder
out Examples:
out # Decode file as vorbis_comment
out $ fq -d vorbis_comment . file
out # Decode value as vorbis_comment
out ... | vorbis_comment
"help(vorbis_packet)"
out vorbis_packet: Vorbis packet decoder
out Examples:
out # Decode file as vorbis_packet
out $ fq -d vorbis_packet . file
out # Decode value as vorbis_packet
out ... | vorbis_packet
"help(vp8_frame)"
out vp8_frame: VP8 frame decoder
out Examples:
out # Decode file as vp8_frame
out $ fq -d vp8_frame . file
out # Decode value as vp8_frame
out ... | vp8_frame
"help(vp9_cfm)"
out vp9_cfm: VP9 Codec Feature Metadata decoder
out Examples:
out # Decode file as vp9_cfm
out $ fq -d vp9_cfm . file
out # Decode value as vp9_cfm
out ... | vp9_cfm
"help(vp9_frame)"
out vp9_frame: VP9 frame decoder
out Examples:
out # Decode file as vp9_frame
out $ fq -d vp9_frame . file
out # Decode value as vp9_frame
out ... | vp9_frame
"help(vpx_ccr)"
out vpx_ccr: VPX Codec Configuration Record decoder
out Examples:
out # Decode file as vpx_ccr
out $ fq -d vpx_ccr . file
out # Decode value as vpx_ccr
out ... | vpx_ccr
"help(wasm)"
out wasm: WebAssembly Binary Format decoder
out Examples:
out # Decode file as wasm
out $ fq -d wasm . file
out # Decode value as wasm
out ... | wasm
"help(wav)"
out wav: WAV file decoder
out Examples:
out # Decode file as wav
out $ fq -d wav . file
out # Decode value as wav
out ... | wav
"help(webp)"
out webp: WebP image decoder
out Examples:
out # Decode file as webp
out $ fq -d webp . file
out # Decode value as webp
out ... | webp
"help(xing)"
out xing: Xing header decoder
out Examples:
out # Decode file as xing
out $ fq -d xing . file
out # Decode value as xing
out ... | xing
"help(xml)"
out xml: Extensible Markup Language decoder
out Options:
out array=false Decode as nested arrays
out attribute_prefix=@ Prefix for attribute keys
out seq=false Use seq attribute to preserve element order
out Examples:
out # Decode file as xml
out $ fq -d xml . file
out # Decode value as xml
out ... | xml
out # Decode file using xml options
out $ fq -d xml -o array=false -o attribute_prefix="@" -o seq=false . file
out # Decode value as xml
out ... | xml({array:false,attribute_prefix:"@",seq:false})
out References and links
out xml.com's Converting Between XML and JSON https://www.xml.com/pub/a/2006/05/31/converting-between-xml-and-json.html
"help(yaml)"
out yaml: YAML Ain't Markup Language decoder
out Examples:
out # Decode file as yaml
out $ fq -d yaml . file
out # Decode value as yaml
out ... | yaml
"help(zip)"
out zip: ZIP archive decoder
out Supports ZIP64.
out Options:
out uncompress=true Uncompress and probe files
out Examples:
out # Decode file as zip
out $ fq -d zip . file
out # Decode value as zip
out ... | zip
out # Decode file using zip options
out $ fq -d zip -o uncompress=true . file
out # Decode value as zip
out ... | zip({uncompress:true})
out References and links
out https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT

11
format/ar/ar.go
View File

@ -27,11 +27,12 @@ func decodeAr(d *decode.D, _ any) any {
for !d.End() {
d.FieldStruct("file", func(d *decode.D) {
d.FieldUTF8("identifier", 16, scalar.ActualTrimSpace)
d.FieldUTF8("modification_timestamp", 12, scalar.ActualTrimSpace, scalar.SymUParseUint(10))
d.FieldUTF8("owner_id", 6, scalar.ActualTrimSpace, scalar.SymUParseUint(10))
d.FieldUTF8("group_id", 6, scalar.ActualTrimSpace, scalar.SymUParseUint(10))
d.FieldUTF8("file_mode", 8, scalar.ActualTrimSpace, scalar.SymUParseUint(8)) // Octal
sizeS := d.FieldScalarUTF8("file_size", 10, scalar.ActualTrimSpace, scalar.SymUParseUint(10))
// TODO: try scalar.DescriptionSymUUnixTime
d.FieldUTF8("modification_timestamp", 12, scalar.ActualTrimSpace, scalar.TrySymUParseUint(10))
d.FieldUTF8("owner_id", 6, scalar.ActualTrimSpace, scalar.TrySymUParseUint(10))
d.FieldUTF8("group_id", 6, scalar.ActualTrimSpace, scalar.TrySymUParseUint(10))
d.FieldUTF8("file_mode", 8, scalar.ActualTrimSpace, scalar.TrySymUParseUint(8)) // Octal
sizeS := d.FieldScalarUTF8("file_size", 10, scalar.ActualTrimSpace, scalar.TrySymUParseUint(10))
if sizeS.Sym == nil {
d.Fatalf("could not decode file_size")
}

3
format/asn1/asn1_ber.go
View File

@ -29,6 +29,7 @@ import (
)
//go:embed asn1_ber.jq
//go:embed asn1_ber.md
var asn1FS embed.FS
func init() {
@ -36,7 +37,7 @@ func init() {
Name: format.ASN1_BER,
Description: "ASN1 BER (basic encoding rules, also CER and DER)",
DecodeFn: decodeASN1BER,
Functions: []string{"torepr", "_help"},
Functions: []string{"torepr"},
})
interp.RegisterFS(asn1FS)
}

21
format/asn1/asn1_ber.jq
View File

@ -6,24 +6,3 @@ def _asn1_ber_torepr:
end
else .constructed | map(_asn1_ber_torepr)
end;
def _asn1_ber__help:
{ notes: "Supports decoding BER, CER and DER (X.690).
- Currently no extra validation is done for CER and DER.
- Does not support specifying a schema.
- Supports `torepr` but without schema all sequences and sets will be arrays.",
examples: [
{comment: "`frompem` and `topem` can be used to work with PEM format", shell: "fq -d raw 'frompem | asn1_ber | d' cert.pem"},
{comment: "Can be used to decode nested parts", shell: "fq -d asn1_ber '.constructed[1].value | asn1_ber' file.ber"},
{ comment: "If schema is known and not complicated it can be reproduced",
shell: "fq -d asn1_ber 'torepr as $r | [\"version\", \"modulus\", \"private_exponent\", \"private_exponen\", \"prime1\", \"prime2\", \"exponent1\", \"exponent2\", \"coefficient\"] | with_entries({key: .value, value: $r[.key]})' pkcs1.der"
}
],
links: [
{url: "https://www.itu.int/ITU-T/studygroups/com10/languages/X.690_1297.pdf"},
{url: "https://en.wikipedia.org/wiki/X.690"},
{url: "https://letsencrypt.org/docs/a-warm-welcome-to-asn1-and-der/"},
{url: "https://lapo.it/asn1js/"}
]
};

29
format/asn1/asn1_ber.md Normal file
View File

@ -0,0 +1,29 @@
Supports decoding BER, CER and DER (X.690).
- Currently no extra validation is done for CER and DER.
- Does not support specifying a schema.
- Supports `torepr` but without schema all sequences and sets will be arrays.
### Can be used to decode certificates etc
```sh
$ fq -d raw 'frompem | asn1_ber | d' cert.pem
```
### Can decode nested values
```sh
$ fq -d asn1_ber '.constructed[1].value | asn1_ber' file.ber
```
### Manual schema
```sh
$ fq -d asn1_ber 'torepr as $r | ["version", "modulus", "private_exponent", "private_exponen", "prime1", "prime2", "exponent1", "exponent2", "coefficient"] | with_entries({key: .value, value: $r[.key]})' pkcs1.der
```
### References
- https://www.itu.int/ITU-T/studygroups/com10/languages/X.690_1297.pdf
- https://en.wikipedia.org/wiki/X.690
- https://letsencrypt.org/docs/a-warm-welcome-to-asn1-and-der/
- https://lapo.it/asn1js/

39
format/asn1/testdata/help_asn1_ber.fqtest vendored Normal file
View File

@ -0,0 +1,39 @@
$ fq -h asn1_ber
asn1_ber: ASN1 BER (basic encoding rules, also CER and DER) decoder
Decode examples
===============
# Decode file as asn1_ber
$ fq -d asn1_ber . file
# Decode value as asn1_ber
... | asn1_ber
Supports decoding BER, CER and DER (X.690).
- Currently no extra validation is done for CER and DER.
- Does not support specifying a schema.
- Supports torepr but without schema all sequences and sets will be arrays.
Can be used to decode certificates etc
======================================
$ fq -d raw 'frompem | asn1_ber | d' cert.pem
Can decode nested values
========================
$ fq -d asn1_ber '.constructed[1].value | asn1_ber' file.ber
Manual schema
=============
$ fq -d asn1_ber 'torepr as $r | ["version", "modulus", "private_exponent", "private_exponen", "prime1", "prime2", "exponent1", "exponent2", "coefficient"] | with_entries({key: .value, value: $r[.key]})' pkcs1.der
References
==========
- https://www.itu.int/ITU-T/studygroups/com10/languages/X.690_1297.pdf
- https://en.wikipedia.org/wiki/X.690
- https://letsencrypt.org/docs/a-warm-welcome-to-asn1-and-der/
- https://lapo.it/asn1js/

3
format/avro/avro_ocf.go
View File

@ -16,7 +16,7 @@ import (
"github.com/wader/fq/pkg/scalar"
)
//go:embed avro_ocf.jq
//go:embed avro_ocf.md
var avroOcfFS embed.FS
func init() {
@ -25,7 +25,6 @@ func init() {
Description: "Avro object container file",
Groups: []string{format.PROBE},
DecodeFn: decodeAvroOCF,
Functions: []string{"_help"},
})
interp.RegisterFS(avroOcfFS)
}

12
format/avro/avro_ocf.jq
View File

@ -1,12 +0,0 @@
def _avro_ocf__help:
{ notes: "Supports reading Avro Object Container Format (OCF) files based on the 1.11.0 specification.
Capable of handling null, deflate, and snappy codecs for data compression.
Limitations:
- Schema does not support self-referential types, only built-in types.
- Decimal logical types are not supported for decoding, will just be treated as their primitive type",
links: [
{url: "https://avro.apache.org/docs/current/spec.html#Object+Container+Files"}
]
};

16
format/avro/avro_ocf.md Normal file
View File

@ -0,0 +1,16 @@
Supports reading Avro Object Container Format (OCF) files based on the 1.11.0 specification.
Capable of handling null, deflate, and snappy codecs for data compression.
Limitations:
- Schema does not support self-referential types, only built-in types.
- Decimal logical types are not supported for decoding, will just be treated as their primitive type
### References
- https://avro.apache.org/docs/current/spec.html#Object+Container+Files
### Authors
- Xentripetal
xentripetal@fastmail.com
[@xentripetal](https://github.com/xentripetal)

28
format/avro/testdata/help_avro_ocf.fqtest vendored Normal file
View File

@ -0,0 +1,28 @@
$ fq -h avro_ocf
avro_ocf: Avro object container file decoder
Decode examples
===============
# Decode file as avro_ocf
$ fq -d avro_ocf . file
# Decode value as avro_ocf
... | avro_ocf
Supports reading Avro Object Container Format (OCF) files based on the 1.11.0 specification.
Capable of handling null, deflate, and snappy codecs for data compression.
Limitations:
- Schema does not support self-referential types, only built-in types.
- Decimal logical types are not supported for decoding, will just be treated as their primitive type
References
==========
- https://avro.apache.org/docs/current/spec.html#Object+Container+Files
Authors
=======
- Xentripetal xentripetal@fastmail.com @xentripetal (https://github.com/xentripetal)

3
format/bencode/bencode.go
View File

@ -13,6 +13,7 @@ import (
)
//go:embed bencode.jq
//go:embed bencode.md
var bencodeFS embed.FS
func init() {
@ -20,7 +21,7 @@ func init() {
Name: format.BENCODE,
Description: "BitTorrent bencoding",
DecodeFn: decodeBencode,
Functions: []string{"torepr", "_help"},
Functions: []string{"torepr"},
})
interp.RegisterFS(bencodeFS)
}

6
format/bencode/bencode.jq
View File

@ -9,9 +9,3 @@ def _bencode_torepr:
)
else error("unknown type \(.type)")
end;
def _bencode__help:
{ links: [
{url: "https://wiki.theory.org/BitTorrentSpecification#Bencoding"}
]
};

8
format/bencode/bencode.md Normal file
View File

@ -0,0 +1,8 @@
### Convert represented value to JSON
```
$ fq -d bencode torepr file.torrent
```
### References
- https://wiki.theory.org/BitTorrentSpecification#Bencoding

20
format/bencode/testdata/help_bencode.fqtest vendored Normal file
View File

@ -0,0 +1,20 @@
$ fq -h bencode
bencode: BitTorrent bencoding decoder
Decode examples
===============
# Decode file as bencode
$ fq -d bencode . file
# Decode value as bencode
... | bencode
Convert represented value to JSON
=================================
$ fq -d bencode torepr file.torrent
References
==========
- https://wiki.theory.org/BitTorrentSpecification#Bencoding

3
format/bitcoin/bitcoin_block.go
View File

@ -4,6 +4,7 @@ package bitcoin
import (
"fmt"
"time"
"github.com/wader/fq/format"
"github.com/wader/fq/pkg/decode"
@ -62,7 +63,7 @@ func decodeBitcoinBlock(d *decode.D, in interface{}) interface{} {
d.FieldU32("version", scalar.ActualHex)
d.FieldRawLen("previous_block_hash", 32*8, rawHexReverse)
d.FieldRawLen("merkle_root", 32*8, rawHexReverse)
d.FieldU32("time", scalar.DescriptionActualUUnixTime)
d.FieldU32("time", scalar.DescriptionUnixTimeFn(scalar.S.TryActualU, time.RFC3339))
d.FieldU32("bits", scalar.ActualHex)
d.FieldU32("nonce", scalar.ActualHex)
})

20
format/bitcoin/testdata/help_bitcoin_block.fqtest vendored Normal file
View File

@ -0,0 +1,20 @@
$ fq -h bitcoin_block
bitcoin_block: Bitcoin block decoder
Options
=======
has_header=false Has blkdat header
Decode examples
===============
# Decode file as bitcoin_block
$ fq -d bitcoin_block . file
# Decode value as bitcoin_block
... | bitcoin_block
# Decode file using bitcoin_block options
$ fq -d bitcoin_block -o has_header=false . file
# Decode value as bitcoin_block
... | bitcoin_block({has_header:false})

292
format/bplist/bplist.go Normal file
View File

@ -0,0 +1,292 @@
package bplist
import (
"embed"
"math"
"math/big"
"time"
"github.com/wader/fq/format"
"github.com/wader/fq/pkg/decode"
"github.com/wader/fq/pkg/interp"
"github.com/wader/fq/pkg/scalar"
)
//go:embed bplist.jq bplist.md
var bplistFS embed.FS
func init() {
interp.RegisterFormat(decode.Format{
Name: format.BPLIST,
ProbeOrder: format.ProbeOrderBinUnique,
Description: "Apple Binary Property List",
Groups: []string{format.PROBE},
DecodeFn: bplistDecode,
Functions: []string{"torepr"},
})
interp.RegisterFS(bplistFS)
}
const (
elementTypeNullOrBoolOrFill = 0x00
elementTypeInt = 0x01
elementTypeReal = 0x02
elementTypeDate = 0x03
elementTypeData = 0x04
elementTypeASCIIString = 0x05
elementTypeUnicodeString = 0x06
elementTypeUID = 0x08
elementTypeArray = 0x0a
elementTypeSet = 0x0c
elementTypeDict = 0x0d
)
const (
null = 0x00
boolFalse = 0x08
boolTrue = 0x09
)
var elementTypeMap = scalar.UToScalar{
elementTypeNullOrBoolOrFill: {Sym: "singleton", Description: "Singleton value (null/bool)"},
elementTypeInt: {Sym: "int", Description: "Integer"},
elementTypeReal: {Sym: "real", Description: "Floating Point Number"},
elementTypeDate: {Sym: "date", Description: "Date, 4 or 8 byte float"},
elementTypeData: {Sym: "data", Description: "Binary data"},
elementTypeASCIIString: {Sym: "ascii_string", Description: "ASCII encoded string"},
elementTypeUnicodeString: {Sym: "unicode_string", Description: "Unicode string"},
elementTypeUID: {Sym: "uid", Description: "UID"},
elementTypeArray: {Sym: "array", Description: "Array"},
elementTypeSet: {Sym: "set", Description: "Set"},
elementTypeDict: {Sym: "dict", Description: "Dictionary"},
}
var cocoaTimeEpochDate = time.Date(2001, time.January, 1, 0, 0, 0, 0, time.UTC)
// decodes the number of bits required to store the following object
func decodeSize(d *decode.D, sms ...scalar.Mapper) uint64 {
n := d.FieldU4("size_bits")
if n != 0x0f {
return n
}
d.FieldU4("large_size_marker", d.AssertU(0b0001))
// get the exponent value
n = d.FieldU4("exponent")
// calculate the number of bytes encoding the size
n = 1 << n
// decode that many bytes as big endian
n = d.FieldUFn(
"size_bigint",
func(d *decode.D) uint64 {
v := d.UBigInt(int(n * 8))
d.AssertBigIntRange(big.NewInt(1), big.NewInt(math.MaxInt64))
return v.Uint64()
}, sms...)
return n
}
// decodeItem decodes an object from the plist, and assumes that the current
// seek position of the *decode.D is an object type tag. Returns a bool
// indicating whether or not a string was decoded, which is necssary for
// checking dictionary key type validity.
func decodeItem(d *decode.D, p *plist) bool {
m := d.FieldU4("type", elementTypeMap)
switch m {
case elementTypeNullOrBoolOrFill:
d.FieldU4("value", scalar.UToScalar{
null: scalar.S{Sym: nil},
boolTrue: scalar.S{Sym: true},
boolFalse: scalar.S{Sym: false},
})
case elementTypeInt:
n := d.FieldUFn("size", func(d *decode.D) uint64 {
return 1 << d.U4()
})
switch n {
case 1:
d.FieldU8("value")
case 2:
d.FieldU16("value")
case 4:
d.FieldU32("value")
case 8:
d.FieldS64("value")
case 16:
d.FieldSBigInt("value", int(n*8))
default:
d.Errorf("invalid integer size %d", n)
}
case elementTypeReal:
n := 1 << decodeSize(d)
d.FieldValueU("size", uint64(n))
d.FieldF("value", n*8)
case elementTypeDate:
n := 1 << decodeSize(d, d.AssertU(4, 8))
d.FieldValueU("size", uint64(n))
d.FieldF("value", n*8, scalar.DescriptionTimeFn(scalar.S.TryActualF, cocoaTimeEpochDate, time.RFC3339))
case elementTypeData:
n := decodeSize(d)
d.FieldValueU("size", n)
d.FieldRawLen("value", int64(n*8))
case elementTypeASCIIString:
n := decodeSize(d)
d.FieldValueU("size", n)
d.FieldUTF8("value", int(n))
return true
case elementTypeUnicodeString:
n := decodeSize(d)
d.FieldValueU("size", n)
d.FieldUTF16("value", int(n))
return true
case elementTypeUID:
n := decodeSize(d)
d.FieldValueU("size", n)
d.FieldUBigInt("value", int(n)).Uint64()
case elementTypeArray:
n := decodeSize(d)
d.FieldValueU("size", n)
d.FieldStructNArray("entries", "entry", int64(n),
func(d *decode.D) {
idx := d.FieldU("object_index", int(p.t.objRefSize)*8)
p.decodeReference(d, idx)
})
case elementTypeSet:
n := decodeSize(d)
d.FieldValueU("size", n)
d.FieldStructNArray("entries", "entry", int64(n),
func(d *decode.D) {
idx := d.FieldU("object_index", int(p.t.objRefSize)*8)
p.decodeReference(d, idx)
})
case elementTypeDict:
n := decodeSize(d)
d.FieldValueU("size", n)
d.FieldStructNArray("entries", "entry", int64(n),
func(d *decode.D) {
var ki, vi uint64
ki = d.FieldU("key_index", int(p.t.objRefSize)*8)
d.SeekRel(int64((n-1)*p.t.objRefSize)*8, func(d *decode.D) {
vi = d.FieldU("value_index", int(p.t.objRefSize)*8)
})
d.FieldStruct("key", func(d *decode.D) {
if k := p.decodeReference(d, ki); !k {
d.Errorf("non-string key in dictionary")
}
})
d.FieldStruct("value", func(d *decode.D) {
p.decodeReference(d, vi)
})
})
default:
d.Errorf("unknown type marker: %d", m)
}
return false
}
// decodeReference looks up and decodes an object based on its index in the
// offset table. Returns a bool indicating whether or not the decoded item is
// a string (necessary for checking dictionary key validity).
func (pl *plist) decodeReference(d *decode.D, idx uint64) bool {
if idx > uint64(len(pl.o)) {
// prevent a panic
d.Errorf("index %d out of bounds for object table size %d", idx, len(pl.o))
return false
}
if pl.indexIsInStack(idx) {
d.Fatalf("recursion detected: object %d already decoded in stack %v", idx, pl.objectStack)
return false
}
pl.pushIndex(idx)
itemOffset := pl.o[idx]
if itemOffset >= pl.t.offsetTableStart {
d.Errorf("attempting to decode object %d at offset 0x%x beyond offset table start 0x%x",
idx, itemOffset, pl.t.offsetTableStart)
}
var isString bool
d.SeekAbs(int64(itemOffset*8), func(d *decode.D) {
isString = decodeItem(d, pl)
})
pl.popIndex()
return isString
}
type trailer struct {
offTblOffSize uint64
objRefSize uint64
nObjects uint64
topObjectOffset uint64
offsetTableStart uint64
}
type plist struct {
t trailer
o []uint64
objectStack []uint64
}
func (pl *plist) pushIndex(idx uint64) {
pl.objectStack = append(pl.objectStack, idx)
}
func (pl *plist) popIndex() {
pl.objectStack = pl.objectStack[:len(pl.objectStack)-1]
}
func (pl *plist) indexIsInStack(idx uint64) bool {
for _, existing := range pl.objectStack {
if existing == idx {
return true
}
}
return false
}
func bplistDecode(d *decode.D, _ any) any {
d.FieldStruct("header", func(d *decode.D) {
d.FieldUTF8("magic", 6, d.AssertStr("bplist"))
d.FieldUTF8("version", 2, d.AssertStr("00"))
})
p := new(plist)
d.SeekAbs(d.Len()-32*8, func(d *decode.D) {
d.FieldStruct("trailer", func(d *decode.D) {
d.FieldU40("unused")
d.FieldS8("sort_version")
p.t.offTblOffSize = d.FieldU8("offset_table_offset_size", d.AssertURange(1, 8))
p.t.objRefSize = d.FieldU8("object_reference_size", d.AssertURange(1, 8))
p.t.nObjects = d.FieldU64("object_count")
p.t.topObjectOffset = d.FieldU64("top_object_offset")
p.t.offsetTableStart = d.FieldU64("offset_table_start")
})
})
d.SeekAbs(int64(p.t.offsetTableStart*8), func(d *decode.D) {
i := uint64(0)
d.FieldArrayLoop("offset_table",
func() bool { return i < p.t.nObjects },
func(d *decode.D) {
off := d.FieldU("element", 8*int(p.t.offTblOffSize))
p.o = append(p.o, off)
i++
},
)
})
d.FieldStruct("objects",
func(d *decode.D) {
p.decodeReference(d, 0)
})
return nil
}

30
format/bplist/bplist.jq Normal file
View File

@ -0,0 +1,30 @@
def _bplist_torepr:
def _f:
( if .type == "singleton" then .value | tovalue
elif .type == "int" then .value | tovalue
elif .type == "real" then .value | tovalue
elif .type == "date" then .value | tovalue
elif .type == "data" then .value | tovalue
elif .type == "ascii_string" then .value | tovalue
elif .type == "unicode_string" then .value | tovalue
elif .type == "uid" then .value | tovalue
elif .type == "array" then
( .entries
| map(_f)
)
elif .type == "set" then
( .entries
| map(_f)
)
elif .type == "dict" then
( .entries
| map({key: (.key | _f), value: (.value | _f)})
| from_entries
)
else error("unknown type: \(.type)")
end
);
( .objects
| _f
);

37
format/bplist/bplist.md Normal file
View File

@ -0,0 +1,37 @@
### Show full decoding
```sh
$ fq d Info.plist
```
### Timestamps
Timestamps in Apple Binary Property Lists are encoded as Cocoa Core Data
timestamps, where the raw value is the floating point number of seconds since
January 1, 2001. By default, `fq` will render the raw floating point value. In
order to get the raw value or string description, use the `todescription`
function, you can use the `tovalue` and `todescription` functions:
```sh
$ fq 'torepr.SomeTimeStamp | tovalue' Info.plist
685135328
$ fq 'torepr.SomeTimeStamp | todescription' Info.plist
"2022-09-17T19:22:08Z"
```
### Get JSON representation
```sh
$ fq torepr com.apple.UIAutomation.plist
{
"UIAutomationEnabled": true
}
```
### Authors
- David McDonald
[@dgmcdona](https://github.com/dgmcdona)
### References
- http://fileformats.archiveteam.org/wiki/Property_List/Binary
- https://medium.com/@karaiskc/understanding-apples-binary-property-list-format-281e6da00dbd
- https://opensource.apple.com/source/CF/CF-550/CFBinaryPList.c

BIN
format/bplist/testdata/Info.plist vendored Normal file
View File

Binary file not shown.

4907
format/bplist/testdata/bplist.fqtest vendored Normal file
View File

File diff suppressed because it is too large Load Diff

BIN
format/bplist/testdata/com.apple.UIAutomation.plist vendored Normal file
View File

Binary file not shown.

BIN
format/bplist/testdata/dicttest_many_items.plist vendored Normal file
View File

Binary file not shown.

BIN
format/bplist/testdata/float.plist vendored Normal file
View File

Binary file not shown.

3
format/bson/bson.go
View File

@ -13,6 +13,7 @@ import (
)
//go:embed bson.jq
//go:embed bson.md
var bsonFS embed.FS
func init() {
@ -20,7 +21,7 @@ func init() {
Name: format.BSON,
Description: "Binary JSON",
DecodeFn: decodeBSON,
Functions: []string{"torepr", "_help"},
Functions: []string{"torepr"},
})
interp.RegisterFS(bsonFS)
}

8
format/bson/bson.jq
View File

@ -17,11 +17,3 @@ def _bson_torepr:
| _f
);
def _bson__help:
{ examples: [
{comment: "BSON as JSON", shell: "fq -d bson torepr file"}
],
links: [
{url: "https://wiki.theory.org/BitTorrentSpecification#Bencoding"}
]
};

14
format/bson/bson.md Normal file
View File

@ -0,0 +1,14 @@
### Convert represented value to JSON
```
$ fq -d bson torepr file.bson
```
### Filter represented value
```
$ fq -d bson 'torepr | select(.name=="bob")' file.bson
```
### References
- https://bsonspec.org/spec.html

25
format/bson/testdata/help_bson.fqtest vendored Normal file
View File

@ -0,0 +1,25 @@
$ fq -h bson
bson: Binary JSON decoder
Decode examples
===============
# Decode file as bson
$ fq -d bson . file
# Decode value as bson
... | bson
Convert represented value to JSON
=================================
$ fq -d bson torepr file.bson
Filter represented value
========================
$ fq -d bson 'torepr | select(.name=="bob")' file.bson
References
==========
- https://bsonspec.org/spec.html

3
format/cbor/cbor.go
View File

@ -21,6 +21,7 @@ import (
)
//go:embed cbor.jq
//go:embed cbor.md
var cborFS embed.FS
func init() {
@ -28,7 +29,7 @@ func init() {
Name: format.CBOR,
Description: "Concise Binary Object Representation",
DecodeFn: decodeCBOR,
Functions: []string{"torepr", "_help"},
Functions: []string{"torepr"},
})
interp.RegisterFS(cborFS)
}

7
format/cbor/cbor.jq
View File

@ -8,10 +8,3 @@ def _cbor_torepr:
elif .major_type == "bytes" then .value | tostring
else .value | tovalue
end;
def _cbor__help:
{ links: [
{url: "https://en.wikipedia.org/wiki/CBOR"},
{url: "https://www.rfc-editor.org/rfc/rfc8949.html"}
]
};

9
format/cbor/cbor.md Normal file
View File

@ -0,0 +1,9 @@
### Convert represented value to JSON
```
$ fq -d cbor torepr file.cbor
```
### References
- https://en.wikipedia.org/wiki/CBOR
- https://www.rfc-editor.org/rfc/rfc8949.html

21
format/cbor/testdata/help_cbor.fqtest vendored Normal file
View File

@ -0,0 +1,21 @@
$ fq -h cbor
cbor: Concise Binary Object Representation decoder
Decode examples
===============
# Decode file as cbor
$ fq -d cbor . file
# Decode value as cbor
... | cbor
Convert represented value to JSON
=================================
$ fq -d cbor torepr file.cbor
References
==========
- https://en.wikipedia.org/wiki/CBOR
- https://www.rfc-editor.org/rfc/rfc8949.html

1
format/csv/csv.go
View File

@ -17,6 +17,7 @@ import (
)
//go:embed csv.jq
//go:embed csv.md
var csvFS embed.FS
func init() {

11
format/csv/csv.md Normal file
View File

@ -0,0 +1,11 @@
### TSV to CSV
```sh
$ fq -d csv -o comma="\t" tocsv file.tsv
```
### Convert rows to objects based on header row
```sh
$ fq -d csv '.[0] as $t | .[1:] | map(with_entries(.key = $t[.key]))' file.csv
```

31
format/csv/testdata/help_csv.fqtest vendored Normal file
View File

@ -0,0 +1,31 @@
$ fq -h csv
csv: Comma separated values decoder
Options
=======
comma=, Separator character
comment=# Comment line character
Decode examples
===============
# Decode file as csv
$ fq -d csv . file
# Decode value as csv
... | csv
# Decode file using csv options
$ fq -d csv -o comma="," -o comment="#" . file
# Decode value as csv
... | csv({comma:",",comment:"#"})
TSV to CSV
==========
$ fq -d csv -o comma="\t" tocsv file.tsv
Convert rows to objects based on header row
===========================================
$ fq -d csv '.[0] as $t | .[1:] | map(with_entries(.key = $t[.key]))' file.csv

29
format/csv/testdata/tsv.fqtest vendored Normal file
View File

@ -0,0 +1,29 @@
/test:
1 2 3
4 5 6
$ fq -d csv -o comma=\\t . test
[
[
"1",
"2",
"3"
],
[
"4",
"5",
"6"
]
]
$ fq -d csv -o comma="\t" . test
[
[
"1",
"2",
"3"
],
[
"4",
"5",
"6"
]
]

20
format/flac/testdata/help_flac_frame.fqtest vendored Normal file
View File

@ -0,0 +1,20 @@
$ fq -h flac_frame
flac_frame: FLAC frame decoder
Options
=======
bits_per_sample=16 Bits per sample
Decode examples
===============
# Decode file as flac_frame
$ fq -d flac_frame . file
# Decode value as flac_frame
... | flac_frame
# Decode file using flac_frame options
$ fq -d flac_frame -o bits_per_sample=16 . file
# Decode value as flac_frame
... | flac_frame({bits_per_sample:16})

15
format/format.go
View File

@ -45,6 +45,7 @@ const (
BITCOIN_BLOCK = "bitcoin_block"
BITCOIN_SCRIPT = "bitcoin_script"
BITCOIN_TRANSACTION = "bitcoin_transaction"
BPLIST = "bplist"
BSD_LOOPBACK_FRAME = "bsd_loopback_frame"
BSON = "bson"
BZIP2 = "bzip2"
@ -86,6 +87,7 @@ const (
JSONL = "jsonl"
MACHO = "macho"
MACHO_FAT = "macho_fat"
MARKDOWN = "markdown"
MATROSKA = "matroska"
MP3 = "mp3"
MP3_FRAME = "mp3_frame"
@ -102,6 +104,7 @@ const (
OPUS_PACKET = "opus_packet"
PCAP = "pcap"
PCAPNG = "pcapng"
PG_BTREE = "pg_btree"
PG_WAL = "pg_wal"
PG_WALPAGE = "pg_wal_page"
PG_MULTIXACTOFF = "pg_multixact_offsets"
@ -322,6 +325,16 @@ type BitCoinBlockIn struct {
}
type PostgresIn struct {
Flavour string `doc:"PostgreSQL flavour: postgres, postgres13, pgpro..."`
Flavour string `doc:"PostgreSQL flavour: postgres14, pgproee14.., postgres10"`
}
type PostgresHeapIn struct {
Flavour string `doc:"PostgreSQL flavour: postgres14, pgproee14.., postgres10"`
PageNumber int `doc:"First page number in file, default is 0"`
SegmentNumber int `doc:"Segment file number (16790.1 is 1), default is 0"`
}
type PostgresWalIn struct {
Flavour string `doc:"PostgreSQL flavour: postgres14, pgproee14.., postgres10"`
Lsn string `doc:"Current LSN for WAL, use \"select pg_current_wal_lsn()\""`
}

3
format/gzip/gzip.go
View File

@ -8,6 +8,7 @@ import (
"compress/flate"
"hash/crc32"
"io"
"time"
"github.com/wader/fq/format"
"github.com/wader/fq/pkg/decode"
@ -74,7 +75,7 @@ func gzDecode(d *decode.D, _ any) any {
hasComment = d.FieldBool("comment")
d.FieldU3("reserved")
})
d.FieldU32("mtime", scalar.DescriptionActualUUnixTime)
d.FieldU32("mtime", scalar.DescriptionUnixTimeFn(scalar.S.TryActualU, time.RFC3339))
switch compressionMethod {
case delfateMethod:
d.FieldU8("extra_flags", deflateExtraFlagsNames)

3
format/macho/macho.go
View File

@ -14,7 +14,7 @@ import (
"github.com/wader/fq/pkg/scalar"
)
//go:embed macho.jq
//go:embed macho.md
var machoFS embed.FS
func init() {
@ -23,7 +23,6 @@ func init() {
Description: "Mach-O macOS executable",
Groups: []string{format.PROBE},
DecodeFn: machoDecode,
Functions: []string{"_help"},
})
interp.RegisterFS(machoFS)
}

9
format/macho/macho.jq
View File

@ -1,9 +0,0 @@
def _macho__help:
{ notes: "Supports decoding vanilla and FAT Mach-O binaries.",
examples: [
{comment: "Select 64bit load segments", shell: "fq '.load_commands[] | select(.cmd==\"segment_64\")' file"}
],
links: [
{url: "https://github.com/aidansteele/osx-abi-macho-file-format-reference"}
]
};

15
format/macho/macho.md Normal file
View File

@ -0,0 +1,15 @@
Supports decoding vanilla and FAT Mach-O binaries.
### Select 64bit load segments
```sh
$ fq '.load_commands[] | select(.cmd=="segment_64")' file
```
### References
- https://github.com/aidansteele/osx-abi-macho-file-format-reference
### Authors
- Sıddık AÇIL
acils@itu.edu.tr
[@Akaame](https://github.com/Akaame)

26
format/macho/testdata/help_macho.fqtest vendored Normal file
View File

@ -0,0 +1,26 @@
$ fq -h macho
macho: Mach-O macOS executable decoder
Decode examples
===============
# Decode file as macho
$ fq -d macho . file
# Decode value as macho
... | macho
Supports decoding vanilla and FAT Mach-O binaries.
Select 64bit load segments
==========================
$ fq '.load_commands[] | select(.cmd=="segment_64")' file
References
==========
- https://github.com/aidansteele/osx-abi-macho-file-format-reference
Authors
=======
- Sıddık AÇIL acils@itu.edu.tr @Akaame (https://github.com/Akaame)

320
format/markdown/markdown.go Normal file
View File

@ -0,0 +1,320 @@
package markdown
import (
"embed"
"fmt"
"io"
"github.com/gomarkdown/markdown"
"github.com/gomarkdown/markdown/ast"
"github.com/wader/fq/format"
"github.com/wader/fq/pkg/bitio"
"github.com/wader/fq/pkg/decode"
"github.com/wader/fq/pkg/interp"
"github.com/wader/fq/pkg/scalar"
)
//go:embed markdown.jq
//go:embed markdown.md
var markdownFS embed.FS
func init() {
interp.RegisterFormat(decode.Format{
Name: format.MARKDOWN,
Description: "Markdown",
DecodeFn: decodeMarkdown,
Functions: []string{"_todisplay"},
})
interp.RegisterFS(markdownFS)
}
func decodeMarkdown(d *decode.D, _ any) any {
b, err := io.ReadAll(bitio.NewIOReader(d.RawLen(d.Len())))
if err != nil {
panic(err)
}
var s scalar.S
s.Actual = node(markdown.Parse(b, nil))
d.Value.V = &s
d.Value.Range.Len = d.Len()
return nil
}
func stringSlice[T string | []byte](ss []T) []any {
var vs []any
for _, e := range ss {
vs = append(vs, string(e))
}
return vs
}
func sliceMap[F, T any](vs []F, fn func(F) T) []T {
ts := make([]T, len(vs))
for i, v := range vs {
ts[i] = fn(v)
}
return ts
}
func intSlice[T ~int](ss []T) []any {
var vs []any
for _, e := range ss {
vs = append(vs, e)
}
return vs
}
func attr(v map[string]any, attr *ast.Attribute) {
if attr == nil {
return
}
v["id"] = string(attr.ID)
var as []any
for _, a := range attr.Attrs {
as = append(as, string(a))
}
v["attrs"] = as
var cs []any
for _, a := range attr.Classes {
cs = append(cs, string(a))
}
v["classes"] = cs
}
func leaf(v map[string]any, typ string, l ast.Leaf) {
v["type"] = typ
v["literal"] = string(l.Literal)
attr(v, l.Attribute)
}
func container(v map[string]any, typ string, c ast.Container) {
v["type"] = typ
v["literal"] = string(c.Literal)
var cs []any
children := c.GetChildren()
for _, n := range children {
cv := node(n)
if cv != nil {
cs = append(cs, node(n))
}
}
v["children"] = cs
attr(v, c.Attribute)
}
func listType(t ast.ListType) []any {
var vs []any
if t&ast.ListTypeOrdered == ast.ListTypeOrdered {
vs = append(vs, "ordered")
}
if t%ast.ListTypeOrdered == ast.ListTypeOrdered {
vs = append(vs, "ordered")
}
if t%ast.ListTypeDefinition == ast.ListTypeDefinition {
vs = append(vs, "definition")
}
if t%ast.ListTypeTerm == ast.ListTypeTerm {
vs = append(vs, "term")
}
if t%ast.ListItemContainsBlock == ast.ListItemContainsBlock {
vs = append(vs, "contains_block")
}
if t%ast.ListItemBeginningOfList == ast.ListItemBeginningOfList {
vs = append(vs, "beginning_of_list")
}
if t%ast.ListItemEndOfList == ast.ListItemEndOfList {
vs = append(vs, "end_of_list")
}
return vs
}
func node(n ast.Node) any {
v := map[string]any{}
switch n := n.(type) {
case *ast.Text:
if n.Leaf.Attribute == nil {
if len(n.Leaf.Literal) > 0 {
return string(n.Leaf.Literal)
}
// skip
return nil
}
case *ast.Softbreak:
leaf(v, "softbreak", n.Leaf)
case *ast.Hardbreak:
leaf(v, "hardbreak", n.Leaf)
case *ast.NonBlockingSpace:
leaf(v, "nbsp", n.Leaf)
case *ast.Emph:
container(v, "em", n.Container)
case *ast.Strong:
container(v, "strong", n.Container)
case *ast.Del:
container(v, "del", n.Container)
case *ast.BlockQuote:
container(v, "blockquote", n.Container)
case *ast.Aside:
container(v, "aside", n.Container)
case *ast.Link:
container(v, "link", n.Container)
v["destination"] = string(n.Destination)
v["title"] = string(n.Title)
v["note_id"] = n.NoteID
v["deferred_id"] = string(n.DeferredID)
v["additional_attributes"] = stringSlice(n.AdditionalAttributes)
case *ast.CrossReference:
container(v, "cross_reference", n.Container)
v["destination"] = string(n.Destination)
case *ast.Citation:
leaf(v, "citation", n.Leaf)
v["destination"] = stringSlice(n.Destination)
v["type"] = sliceMap(n.Type, func(v ast.CitationTypes) string {
switch v {
case ast.CitationTypeNone:
return "none"
case ast.CitationTypeSuppressed:
return "suppressed"
case ast.CitationTypeInformative:
return "informative"
case ast.CitationTypeNormative:
return "normative"
default:
return "unknown"
}
})
v["type"] = intSlice(n.Type)
v["suffix"] = stringSlice(n.Suffix)
case *ast.Image:
container(v, "image", n.Container)
v["destination"] = string(n.Destination)
v["title"] = string(n.Title)
case *ast.Code:
leaf(v, "code", n.Leaf)
case *ast.CodeBlock:
leaf(v, "code_block", n.Leaf)
v["is_fenced"] = n.IsFenced
v["info"] = string(n.Info)
if n.FenceChar != 0 {
v["fence_char"] = string(n.FenceChar)
}
v["fence_length"] = n.FenceLength
v["fence_offset"] = n.FenceOffset
case *ast.Caption:
container(v, "caption", n.Container)
case *ast.CaptionFigure:
container(v, "caption_figure", n.Container)
v["heading_id"] = n.HeadingID
case *ast.Document:
container(v, "document", n.Container)
case *ast.Paragraph:
container(v, "paragraph", n.Container)
case *ast.HTMLSpan:
leaf(v, "html_span", n.Leaf)
case *ast.HTMLBlock:
leaf(v, "html_block", n.Leaf)
case *ast.Heading:
container(v, "heading", n.Container)
v["level"] = n.Level
v["heading_id"] = n.HeadingID
v["is_titleblock"] = n.IsTitleblock
v["is_special"] = n.IsSpecial
case *ast.HorizontalRule:
leaf(v, "hr", n.Leaf)
case *ast.List:
container(v, "list", n.Container)
v["list_flags"] = listType(n.ListFlags)
v["tight"] = n.Tight
if n.BulletChar != 0 {
v["bullet_char"] = string(n.BulletChar)
}
if n.Delimiter != 0 {
v["delimiter"] = string(n.Delimiter)
}
v["start"] = n.Start
v["ref_link"] = string(n.RefLink)
v["is_footnotes_list"] = n.IsFootnotesList
case *ast.ListItem:
container(v, "list_item", n.Container)
v["list_flags"] = listType(n.ListFlags)
v["tight"] = n.Tight
if n.BulletChar != 0 {
v["bullet_char"] = string(n.BulletChar)
}
if n.Delimiter != 0 {
v["delimiter"] = string(n.Delimiter)
}
v["ref_link"] = string(n.RefLink)
v["is_footnotes_list"] = n.IsFootnotesList
case *ast.Table:
container(v, "table", n.Container)
case *ast.TableCell:
container(v, "table_cell", n.Container)
v["is_header"] = n.IsHeader
v["align"] = n.Align.String()
v["col_span"] = n.ColSpan
case *ast.TableHeader:
container(v, "table_header", n.Container)
case *ast.TableBody:
container(v, "table_body", n.Container)
case *ast.TableRow:
container(v, "table_row", n.Container)
case *ast.TableFooter:
container(v, "table_footer", n.Container)
case *ast.Math:
leaf(v, "math", n.Leaf)
case *ast.MathBlock:
container(v, "math_block", n.Container)
case *ast.DocumentMatter:
container(v, "document_matter", n.Container)
v["matter"] = func(v ast.DocumentMatters) string {
switch v {
case ast.DocumentMatterNone:
return "none"
case ast.DocumentMatterFront:
return "front"
case ast.DocumentMatterMain:
return "main"
case ast.DocumentMatterBack:
return "back"
default:
return "unknown"
}
}(n.Matter)
case *ast.Callout:
leaf(v, "callout", n.Leaf)
v["id"] = string(n.ID)
case *ast.Index:
leaf(v, "index", n.Leaf)
v["primary"] = n.Primary
v["item"] = string(n.Item)
v["subitem"] = string(n.Subitem)
v["id"] = n.ID
case *ast.Subscript:
leaf(v, "subscript", n.Leaf)
case *ast.Superscript:
leaf(v, "superscript", n.Leaf)
case *ast.Footnotes:
container(v, "footnotes", n.Container)
default:
panic(fmt.Sprintf("unknown node %T", node))
}
for k, e := range v {
if s, ok := e.(string); ok && s == "" {
delete(v, k)
}
}
return v
}

58
format/markdown/markdown.jq Normal file
View File

@ -0,0 +1,58 @@
def _markdown__todisplay: tovalue;
def word_break($width):
def _f($a; $acc; $l):
( $a[0] as $w
| ($w // "" | length) as $wl
| if $w == null then $acc
elif ($l + $wl) >= $width then
( $acc
, _f($a[1:]; [$w]; $wl)
)
else _f($a[1:]; $acc+[$w]; $l+$wl)
end
);
( [_f([splits("\\s{1,}")]; []; 0)]
| map(join(" "))
);
def _markdown_to_text($width; $header_depth):
def lb: if $width > 0 then word_break($width) | join("\n") end;
def _f($pln):
if type == "string" then gsub("\n"; " ")
elif .type == "document" then .children[] | _f("\n\n")
elif .type == "heading" then
( (.children[] | _f("\n\n")) as $title
| $title
, "\n"
, ("=" * ($title | length))
, "\n"
)
elif .type == "paragraph" then
( ( [.children[] | _f("\n\n")]
| join("")
| lb
)
, $pln
)
elif .type == "link" then
( ( [ .children[]
| _f("")
]
| join("")
) as $text
| $text
, if $text != .destination then " (", .destination, ")"
else empty
end
)
elif .type == "code_block" then "\n", (" ", .literal | split("\n") | join("\n ")), "\n"
elif .type == "code" then .literal
elif .type == "list" then (.children[] | _f("\n\n")), "\n" # TODO: delim
elif .type == "list_item" then .bullet_char, " ", (.children[] | _f("\n"))
elif .type == "html_span" then .literal | gsub("<br>"; "\n") # TODO: more?
else empty
end;
[_f("\n\n")] | join("");
def _markdown_to_text:
_markdown_to_text(-1; 0);

4
format/markdown/markdown.md Normal file
View File

@ -0,0 +1,4 @@
### Array with all level 1 and 2 headers
```sh
$ fq -d markdown '[.. | select(.type=="heading" and .level<=2)?.children[0]]' file.md
```

16
format/markdown/testdata/help_markdown.fqtest vendored Normal file
View File

@ -0,0 +1,16 @@
$ fq -h markdown
markdown: Markdown decoder
Decode examples
===============
# Decode file as markdown
$ fq -d markdown . file
# Decode value as markdown
... | markdown
Array with all level 1 and 2 headers
====================================
$ fq -d markdown '[.. | select(.type=="heading" and .level<=2)?.children[0]]' file.md

316
format/markdown/testdata/test.fqtest vendored Normal file
View File

@ -0,0 +1,316 @@
$ fq -d markdown . test.md
{
"children": [
{
"children": [
"Before"
],
"type": "paragraph"
},
{
"children": [
"header 1"
],
"is_special": false,
"is_titleblock": false,
"level": 1,
"type": "heading"
},
{
"children": [
"Paragraph with ",
{
"children": [
"bold"
],
"type": "strong"
},
" and ",
{
"children": [
"italic"
],
"type": "em"
},
"\non\nmultiple\nlines."
],
"type": "paragraph"
},
{
"children": [
{
"children": [
"Some citation"
],
"type": "paragraph"
}
],
"type": "blockquote"
},
{
"children": [
"A footnote",
{
"additional_attributes": [],
"children": [
"^1"
],
"destination": "footnote1",
"note_id": 0,
"type": "link"
},
" and this also",
{
"additional_attributes": [],
"children": [
"^note"
],
"destination": "footnote2",
"note_id": 0,
"type": "link"
}
],
"type": "paragraph"
},
{
"children": [
"header 2"
],
"is_special": false,
"is_titleblock": false,
"level": 2,
"type": "heading"
},
{
"fence_length": 0,
"fence_offset": 0,
"info": "jq",
"is_fenced": true,
"literal": "code\nblock\n",
"type": "code_block"
},
{
"fence_length": 0,
"fence_offset": 0,
"is_fenced": false,
"literal": "also\ncode\n",
"type": "code_block"
},
{
"children": [
"header 3"
],
"is_special": false,
"is_titleblock": false,
"level": 3,
"type": "heading"
},
{
"children": [
"Some text with ",
{
"literal": "code",
"type": "code"
}
],
"type": "paragraph"
},
{
"children": [
"header 4"
],
"is_special": false,
"is_titleblock": false,
"level": 4,
"type": "heading"
},
{
"children": [
"Some text ",
{
"additional_attributes": [],
"children": [
"with a link"
],
"destination": "http://host/path",
"note_id": 0,
"type": "link"
}
],
"type": "paragraph"
},
{
"children": [
"An image ",
{
"children": [
"img alt text"
],
"destination": "path/image.png",
"type": "image"
}
],
"type": "paragraph"
},
{
"children": [
"header 5"
],
"is_special": false,
"is_titleblock": false,
"level": 5,
"type": "heading"
},
{
"children": [
{
"bullet_char": "-",
"children": [
{
"children": [
"list of"
],
"type": "paragraph"
}
],
"delimiter": ".",
"is_footnotes_list": false,
"list_flags": [],
"tight": false,
"type": "list_item"
},
{
"bullet_char": "-",
"children": [
{
"children": [
"things"
],
"type": "paragraph"
}
],
"delimiter": ".",
"is_footnotes_list": false,
"list_flags": [],
"tight": false,
"type": "list_item"
}
],
"delimiter": ".",
"is_footnotes_list": false,
"list_flags": [],
"start": 0,
"tight": true,
"type": "list"
},
{
"children": [
"a table"
],
"type": "paragraph"
},
{
"children": [
{
"children": [
{
"children": [
{
"children": [
"a"
],
"col_span": 0,
"is_header": true,
"type": "table_cell"
},
{
"children": [
"b"
],
"col_span": 0,
"is_header": true,
"type": "table_cell"
},
{
"children": [
"c"
],
"col_span": 0,
"is_header": true,
"type": "table_cell"
}
],
"type": "table_row"
}
],
"type": "table_header"
},
{
"children": [
{
"children": [
{
"children": [
"1"
],
"col_span": 0,
"is_header": false,
"type": "table_cell"
},
{
"children": [
"2"
],
"col_span": 0,
"is_header": false,
"type": "table_cell"
},
{
"children": [
"3"
],
"col_span": 0,
"is_header": false,
"type": "table_cell"
}
],
"type": "table_row"
}
],
"type": "table_body"
}
],
"type": "table"
},
{
"children": [
"header 6"
],
"is_special": false,
"is_titleblock": false,
"level": 6,
"type": "heading"
},
{
"children": [
"Some text with line ",
{
"literal": "<br>",
"type": "html_span"
},
" break and ",
{
"literal": "<b>",
"type": "html_span"
},
"bold",
{
"literal": "</b>",
"type": "html_span"
}
],
"type": "paragraph"
}
],
"type": "document"
}

50
format/markdown/testdata/test.md vendored Normal file
View File

@ -0,0 +1,50 @@
Before
# header 1
Paragraph with **bold** and *italic*
on
multiple
lines.
> Some citation
A footnote[^1] and this also[^note]
## header 2
```jq
code
block
```
also
code
### header 3
Some text with `code`
#### header 4
Some text [with a link](http://host/path)
An image ![img alt text](path/image.png)
##### header 5
- list of
- things
a table
| a | b | c |
| --- | --- | --- |
| 1 | 2 | 3 |
###### header 6
Some text with line <br> break and <b>bold</b>
[^1]: footnote1
[^note]: footnote2

2
format/matroska/ebml_matroska/ebml_matroska.go
View File

@ -1,5 +1,5 @@
//nolint:revive
package ebml_matroska
// https://raw.githubusercontent.com/cellar-wg/matroska-specification/aa2144a58b661baf54b99bab41113d66b0f5ff62/ebml_matroska.xml
// https://raw.githubusercontent.com/ietf-wg-cellar/matroska-specification/master/ebml_matroska.xml
//go:generate sh -c "go run ../ebml/gen/main.go ebml_matroska.xml ebml_matroska github.com/wader/fq/format/matroska/ebml github.com/wader/fq/pkg/scalar Segment | gofmt -s > ebml_matroska_gen.go"

346
format/matroska/ebml_matroska/ebml_matroska.xml
View File

@ -4,11 +4,10 @@
<element name="EBMLMaxSizeLength" path="\EBML\EBMLMaxSizeLength" id="0x42F3" type="uinteger" range="1-8" default="8" minOccurs="1" maxOccurs="1"/>
<!-- Root Element-->
<element name="Segment" path="\Segment" id="0x18538067" type="master" minOccurs="1" maxOccurs="1" unknownsizeallowed="1">
<documentation lang="en" purpose="definition">The Root Element that contains all other Top-Level Elements (Elements defined only at Level 1).
A Matroska file is composed of 1 Segment.</documentation>
<documentation lang="en" purpose="definition">The Root Element that contains all other Top-Level Elements; see (#data-layout).</documentation>
</element>
<element name="SeekHead" path="\Segment\SeekHead" id="0x114D9B74" type="master" maxOccurs="2">
<documentation lang="en" purpose="definition">Contains the Segment Position of other Top-Level Elements.</documentation>
<documentation lang="en" purpose="definition">Contains seeking information of Top-Level Elements; see (#data-layout).</documentation>
<extension type="webmproject.org" webm="1"/>
</element>
<element name="Seek" path="\Segment\SeekHead\Seek" id="0x4DBB" type="master" minOccurs="1">
@ -16,48 +15,55 @@ A Matroska file is composed of 1 Segment.</documentation>
<extension type="webmproject.org" webm="1"/>
</element>
<element name="SeekID" path="\Segment\SeekHead\Seek\SeekID" id="0x53AB" type="binary" length="&lt;= 4" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">The binary ID corresponding to the Element name.</documentation>
<documentation lang="en" purpose="definition">The binary EBML ID of a Top-Level Element.</documentation>
<extension type="webmproject.org" webm="1"/>
</element>
<element name="SeekPosition" path="\Segment\SeekHead\Seek\SeekPosition" id="0x53AC" type="uinteger" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">The Segment Position of the Element.</documentation>
<documentation lang="en" purpose="definition">The Segment Position ((#segment-position)) of a Top-Level Element.</documentation>
<extension type="webmproject.org" webm="1"/>
</element>
<element name="Info" path="\Segment\Info" id="0x1549A966" type="master" minOccurs="1" maxOccurs="1" recurring="1">
<documentation lang="en" purpose="definition">Contains general information about the Segment.</documentation>
<extension type="webmproject.org" webm="1"/>
</element>
<element name="SegmentUID" path="\Segment\Info\SegmentUID" id="0x73A4" type="binary" range="not 0" length="16" maxOccurs="1">
<documentation lang="en" purpose="definition">A randomly generated unique ID to identify the Segment amongst many others (128 bits).</documentation>
<element name="SegmentUUID" path="\Segment\Info\SegmentUUID" id="0x73A4" type="binary" range="not 0" length="16" maxOccurs="1">
<documentation lang="en" purpose="definition">A randomly generated unique ID to identify the Segment amongst many others (128 bits).
It is effectively a Universally Unique IDentifier stored in binary form [@!RFC4122].</documentation>
<documentation lang="en" purpose="usage notes">If the Segment is a part of a Linked Segment, then this Element is **REQUIRED**.</documentation>
<extension type="libmatroska" cppname="SegmentUID"/>
</element>
<element name="SegmentFilename" path="\Segment\Info\SegmentFilename" id="0x7384" type="utf-8" maxOccurs="1">
<documentation lang="en" purpose="definition">A filename corresponding to this Segment.</documentation>
</element>
<element name="PrevUID" path="\Segment\Info\PrevUID" id="0x3CB923" type="binary" length="16" maxOccurs="1">
<documentation lang="en" purpose="definition">A unique ID to identify the previous Segment of a Linked Segment (128 bits).</documentation>
<documentation lang="en" purpose="usage notes">If the Segment is a part of a Linked Segment that uses Hard Linking,
then either the PrevUID or the NextUID Element is **REQUIRED**. If a Segment contains a PrevUID but not a NextUID,
then it **MAY** be considered as the last Segment of the Linked Segment. The PrevUID **MUST NOT** be equal to the SegmentUID.</documentation>
<element name="PrevUUID" path="\Segment\Info\PrevUUID" id="0x3CB923" type="binary" length="16" maxOccurs="1">
<documentation lang="en" purpose="definition">A unique ID to identify the previous Segment of a Linked Segment (128 bits).
Like the SegmentUUID, it is a Universally Unique IDentifier stored in binary form [@!RFC4122].</documentation>
<documentation lang="en" purpose="usage notes">If the Segment is a part of a Linked Segment that uses Hard Linking ((#hard-linking)),
then either the PrevUUID or the NextUUID Element is **REQUIRED**. If a Segment contains a PrevUUID but not a NextUUID,
then it **MAY** be considered as the last Segment of the Linked Segment. The PrevUUID **MUST NOT** be equal to the SegmentUUID.</documentation>
<extension type="libmatroska" cppname="PrevUID"/>
</element>
<element name="PrevFilename" path="\Segment\Info\PrevFilename" id="0x3C83AB" type="utf-8" maxOccurs="1">
<documentation lang="en" purpose="definition">A filename corresponding to the file of the previous Linked Segment.</documentation>
<documentation lang="en" purpose="usage notes">Provision of the previous filename is for display convenience,
but PrevUID **SHOULD** be considered authoritative for identifying the previous Segment in a Linked Segment.</documentation>
but PrevUUID **SHOULD** be considered authoritative for identifying the previous Segment in a Linked Segment.</documentation>
</element>
<element name="NextUID" path="\Segment\Info\NextUID" id="0x3EB923" type="binary" length="16" maxOccurs="1">
<documentation lang="en" purpose="definition">A unique ID to identify the next Segment of a Linked Segment (128 bits).</documentation>
<documentation lang="en" purpose="usage notes">If the Segment is a part of a Linked Segment that uses Hard Linking,
then either the PrevUID or the NextUID Element is **REQUIRED**. If a Segment contains a NextUID but not a PrevUID,
then it **MAY** be considered as the first Segment of the Linked Segment. The NextUID **MUST NOT** be equal to the SegmentUID.</documentation>
<element name="NextUUID" path="\Segment\Info\NextUUID" id="0x3EB923" type="binary" length="16" maxOccurs="1">
<documentation lang="en" purpose="definition">A unique ID to identify the next Segment of a Linked Segment (128 bits).
Like the SegmentUUID, it is a Universally Unique IDentifier stored in binary form [@!RFC4122].</documentation>
<documentation lang="en" purpose="usage notes">If the Segment is a part of a Linked Segment that uses Hard Linking ((#hard-linking)),
then either the PrevUUID or the NextUUID Element is **REQUIRED**. If a Segment contains a NextUUID but not a PrevUUID,
then it **MAY** be considered as the first Segment of the Linked Segment. The NextUUID **MUST NOT** be equal to the SegmentUUID.</documentation>
<extension type="libmatroska" cppname="NextUID"/>
</element>
<element name="NextFilename" path="\Segment\Info\NextFilename" id="0x3E83BB" type="utf-8" maxOccurs="1">
<documentation lang="en" purpose="definition">A filename corresponding to the file of the next Linked Segment.</documentation>
<documentation lang="en" purpose="usage notes">Provision of the next filename is for display convenience,
but NextUID **SHOULD** be considered authoritative for identifying the Next Segment.</documentation>
but NextUUID **SHOULD** be considered authoritative for identifying the Next Segment.</documentation>
</element>
<element name="SegmentFamily" path="\Segment\Info\SegmentFamily" id="0x4444" type="binary" length="16">
<documentation lang="en" purpose="definition">A randomly generated unique ID that all Segments of a Linked Segment **MUST** share (128 bits).</documentation>
<documentation lang="en" purpose="definition">A randomly generated unique ID that all Segments of a Linked Segment **MUST** share (128 bits).
It is effectively a Universally Unique IDentifier stored in binary form [@!RFC4122].</documentation>
<documentation lang="en" purpose="usage notes">If the Segment Info contains a `ChapterTranslate` element, this Element is **REQUIRED**.</documentation>
</element>
<element name="ChapterTranslate" path="\Segment\Info\ChapterTranslate" id="0x6924" type="master">
@ -133,7 +139,7 @@ It is useful when using overlay tracks on seeking or to decide what track to use
It could change later if not specified as silent in a further Cluster.</documentation>
<extension type="libmatroska" cppname="ClusterSilentTrackNumber"/>
</element>
<element name="Position" path="\Segment\Cluster\Position" id="0xA7" type="uinteger" maxOccurs="1">
<element name="Position" path="\Segment\Cluster\Position" id="0xA7" type="uinteger" maxver="4" maxOccurs="1">
<documentation lang="en" purpose="definition">The Segment Position of the Cluster in the Segment (0 in live streams).
It might help to resynchronise offset on damaged streams.</documentation>
<extension type="libmatroska" cppname="ClusterPosition"/>
@ -163,7 +169,7 @@ see (#block-structure) on Block Structure.</documentation>
</documentation>
</element>
<element name="BlockAdditions" path="\Segment\Cluster\BlockGroup\BlockAdditions" id="0x75A1" type="master" maxOccurs="1">
<documentation lang="en" purpose="definition">Contain additional blocks to complete the main one.
<documentation lang="en" purpose="definition">Contain additional binary data to complete the main one; see Codec BlockAdditions section of [@?MatroskaCodec] for more information.
An EBML parser that has no knowledge of the Block structure could still see and use/skip these data.</documentation>
<extension type="webmproject.org" webm="1"/>
</element>
@ -171,15 +177,18 @@ An EBML parser that has no knowledge of the Block structure could still see and
<documentation lang="en" purpose="definition">Contain the BlockAdditional and some parameters.</documentation>
<extension type="webmproject.org" webm="1"/>
</element>
<element name="BlockAddID" path="\Segment\Cluster\BlockGroup\BlockAdditions\BlockMore\BlockAddID" id="0xEE" type="uinteger" range="not 0" default="1" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">An ID to identify the BlockAdditional level.
If BlockAddIDType of the corresponding block is 0, this value is also the value of BlockAddIDType for the meaning of the content of BlockAdditional.</documentation>
<extension type="webmproject.org" webm="1"/>
</element>
<element name="BlockAdditional" path="\Segment\Cluster\BlockGroup\BlockAdditions\BlockMore\BlockAdditional" id="0xA5" type="binary" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">Interpreted by the codec as it wishes (using the BlockAddID).</documentation>
<extension type="webmproject.org" webm="1"/>
</element>
<element name="BlockAddID" path="\Segment\Cluster\BlockGroup\BlockAdditions\BlockMore\BlockAddID" id="0xEE" type="uinteger" range="not 0" default="1" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">An ID to identify how to interpret the BlockAdditional data; see Codec BlockAdditions section of [@?MatroskaCodec] for more information.
A value of 1 indicates that the meaning of the BlockAdditional data is defined by the codec.
Any other value indicates the meaning of the BlockAdditional data is found in the BlockAddIDType found in the TrackEntry.</documentation>
<documentation lang="en" purpose="usage notes">Each BlockAddID value **MUST** be unique between all BlockMore elements found in a BlockAdditions.</documentation>
<documentation lang="en" purpose="usage notes">To keep MaxBlockAdditionID as low as possible, small values **SHOULD** be used.</documentation>
<extension type="webmproject.org" webm="1"/>
</element>
<element name="BlockDuration" path="\Segment\Cluster\BlockGroup\BlockDuration" id="0x9B" type="uinteger" maxOccurs="1">
<documentation lang="en" purpose="definition">The duration of the Block, expressed in Track Ticks; see (#timestamp-ticks).
The BlockDuration Element can be useful at the end of a Track to define the duration of the last frame (as there is no subsequent Block available),
@ -217,14 +226,14 @@ This information **SHOULD** always be referenced by a seek entry.</documentation
The duration of DiscardPadding is not calculated in the duration of the TrackEntry and **SHOULD** be discarded during playback.</documentation>
<extension type="webmproject.org" webm="1"/>
</element>
<element name="Slices" path="\Segment\Cluster\BlockGroup\Slices" id="0x8E" type="master" minver="0" maxver="0" maxOccurs="1">
<element name="Slices" path="\Segment\Cluster\BlockGroup\Slices" id="0x8E" type="master" minver="0" maxver="0" maxOccurs="1">
<documentation lang="en" purpose="definition">Contains slices description.</documentation>
</element>
<element name="TimeSlice" path="\Segment\Cluster\BlockGroup\Slices\TimeSlice" id="0xE8" type="master" minver="0" maxver="0">
<documentation lang="en" purpose="definition">Contains extra time information about the data contained in the Block.
Being able to interpret this Element is not **REQUIRED** for playback.</documentation>
</element>
<element name="LaceNumber" path="\Segment\Cluster\BlockGroup\Slices\TimeSlice\LaceNumber" id="0xCC" type="uinteger" minver="0" maxver="0" maxOccurs="1">
<element name="LaceNumber" path="\Segment\Cluster\BlockGroup\Slices\TimeSlice\LaceNumber" id="0xCC" type="uinteger" minver="0" maxver="0" maxOccurs="1">
<documentation lang="en" purpose="definition">The reverse number of the frame in the lace (0 is the last frame, 1 is the next to last, etc).
Being able to interpret this Element is not **REQUIRED** for playback.</documentation>
<extension type="libmatroska" cppname="SliceLaceNumber"/>
@ -374,7 +383,7 @@ see (#defaultdecodedfieldduration) for more information</documentation>
<extension type="stream copy" keep="1"/>
</element>
<element name="TrackTimestampScale" path="\Segment\Tracks\TrackEntry\TrackTimestampScale" id="0x23314F" type="float" maxver="3" range="&gt; 0x0p+0" default="0x1p+0" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">DEPRECATED, DO NOT USE. The scale to apply on this track to work at normal speed in relation with other tracks
<documentation lang="en" purpose="definition">The scale to apply on this track to work at normal speed in relation with other tracks
(mostly used to adjust video speed when the audio length differs).</documentation>
<extension type="libmatroska" cppname="TrackTimecodeScale"/>
<extension type="stream copy" keep="1"/>
@ -393,10 +402,10 @@ A value 0 means there is no BlockAdditions ((#blockadditions-element)) for this
with BlockAddID ((#blockaddid-element)), or to the track as a whole
with BlockAddIDExtraData.</documentation>
</element>
<element name="BlockAddIDValue" path="\Segment\Tracks\TrackEntry\BlockAdditionMapping\BlockAddIDValue" id="0x41F0" type="uinteger" minver="4" range=">=2" maxOccurs="1">
<element name="BlockAddIDValue" path="\Segment\Tracks\TrackEntry\BlockAdditionMapping\BlockAddIDValue" id="0x41F0" type="uinteger" minver="4" range="&gt;=2" maxOccurs="1">
<documentation lang="en" purpose="definition">If the track format extension needs content beside frames,
the value refers to the BlockAddID ((#blockaddid-element)), value being described.
To keep MaxBlockAdditionID as low as possible, small values **SHOULD** be used.</documentation>
the value refers to the BlockAddID ((#blockaddid-element)), value being described.</documentation>
<documentation lang="en" purpose="usage notes">To keep MaxBlockAdditionID as low as possible, small values **SHOULD** be used.</documentation>
</element>
<element name="BlockAddIDName" path="\Segment\Tracks\TrackEntry\BlockAdditionMapping\BlockAddIDName" id="0x41A4" type="string" minver="4" maxOccurs="1">
<documentation lang="en" purpose="definition">A human-friendly name describing the type of BlockAdditional data,
@ -405,6 +414,7 @@ as defined by the associated Block Additional Mapping.</documentation>
<element name="BlockAddIDType" path="\Segment\Tracks\TrackEntry\BlockAdditionMapping\BlockAddIDType" id="0x41E7" type="uinteger" minver="4" default="0" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">Stores the registered identifier of the Block Additional Mapping
to define how the BlockAdditional data should be handled.</documentation>
<documentation lang="en" purpose="usage notes">If BlockAddIDType is 0, the BlockAddIDValue and corresponding BlockAddID values **MUST** be 1.</documentation>
</element>
<element name="BlockAddIDExtraData" path="\Segment\Tracks\TrackEntry\BlockAdditionMapping\BlockAddIDExtraData" id="0x41ED" type="binary" minver="4" maxOccurs="1">
<documentation lang="en" purpose="definition">Extra binary data that the BlockAddIDType can use to interpret the BlockAdditional data.
@ -416,20 +426,21 @@ The interpretation of the binary data depends on the BlockAddIDType value and th
<extension type="webmproject.org" webm="1"/>
</element>
<element name="Language" path="\Segment\Tracks\TrackEntry\Language" id="0x22B59C" type="string" default="eng" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">Specifies the language of the track in the Matroska languages form;
see (#language-codes) on language codes.
This Element **MUST** be ignored if the LanguageIETF Element is used in the same TrackEntry.</documentation>
<documentation lang="en" purpose="definition">The language of the track,
in the Matroska languages form; see (#language-codes) on language codes.
This Element **MUST** be ignored if the LanguageBCP47 Element is used in the same TrackEntry.</documentation>
<extension type="libmatroska" cppname="TrackLanguage"/>
<extension type="webmproject.org" webm="1"/>
</element>
<element name="LanguageIETF" path="\Segment\Tracks\TrackEntry\LanguageIETF" id="0x22B59D" type="string" minver="4" maxOccurs="1">
<documentation lang="en" purpose="definition">Specifies the language of the track according to [@!BCP47]
and using the IANA Language Subtag Registry [@!IANALangRegistry].
<element name="LanguageBCP47" path="\Segment\Tracks\TrackEntry\LanguageBCP47" id="0x22B59D" type="string" minver="4" maxOccurs="1">
<documentation lang="en" purpose="definition">The language of the track,
in the [@!BCP47] form; see (#language-codes) on language codes.
If this Element is used, then any Language Elements used in the same TrackEntry **MUST** be ignored.</documentation>
<extension type="libmatroska" cppname="LanguageIETF"/>
</element>
<element name="CodecID" path="\Segment\Tracks\TrackEntry\CodecID" id="0x86" type="string" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">An ID corresponding to the codec,
see [@!MatroskaCodec] for more info.</documentation>
see [@?MatroskaCodec] for more info.</documentation>
<extension type="stream copy" keep="1"/>
<extension type="webmproject.org" webm="1"/>
</element>
@ -596,8 +607,9 @@ Undefined values **SHOULD NOT** be used as the behavior of known implementations
<extension type="libmatroska" cppname="VideoAlphaMode"/>
<extension type="stream copy" keep="1"/>
</element>
<element name="OldStereoMode" path="\Segment\Tracks\TrackEntry\Video\OldStereoMode" id="0x53B9" type="uinteger" maxver="0" maxOccurs="1">
<documentation lang="en" purpose="definition">DEPRECATED, DO NOT USE. Bogus StereoMode value used in old versions of libmatroska.</documentation>
<element name="OldStereoMode" path="\Segment\Tracks\TrackEntry\Video\OldStereoMode" id="0x53B9" type="uinteger" maxver="2" maxOccurs="1">
<documentation lang="en" purpose="definition">Bogus StereoMode value used in old versions of libmatroska.</documentation>
<documentation lang="en" purpose="usage notes">This Element **MUST NOT** be used. It was an incorrect value used in libmatroska up to 0.9.0.</documentation>
<restriction>
<enum value="0" label="mono"/>
<enum value="1" label="right eye"/>
@ -682,7 +694,6 @@ PixelHeight - PixelCropTop - PixelCropBottom, else there is no default value.</i
<documentation lang="en" purpose="definition">Specify the uncompressed pixel format used for the Track's data as a FourCC.
This value is similar in scope to the biCompression value of AVI's `BITMAPINFO` [@?AVIFormat]. See the YUV video formats [@?FourCC-YUV] and RGB video formats [@?FourCC-RGB] for common values.</documentation>
<implementation_note note_attribute="minOccurs">UncompressedFourCC **MUST** be set (minOccurs=1) in TrackEntry, when the CodecID Element of the TrackEntry is set to "V_UNCOMPRESSED".</implementation_note>
<documentation lang="en" purpose="usage notes">This Element **MUST NOT** be used if the CodecID Element of the TrackEntry is set to "V_UNCOMPRESSED".</documentation>
<extension type="libmatroska" cppname="VideoColourSpace"/>
<extension type="stream copy" keep="1"/>
</element>
@ -800,7 +811,7 @@ the value and meanings for TransferCharacteristics are adopted from Table 3 of I
<enum value="0" label="reserved"/>
<enum value="1" label="ITU-R BT.709"/>
<enum value="2" label="unspecified"/>
<enum value="3" label="reserved"/>
<enum value="3" label="reserved2"/>
<enum value="4" label="Gamma 2.2 curve - BT.470M"/>
<enum value="5" label="Gamma 2.8 curve - BT.470BG"/>
<enum value="6" label="SMPTE 170M"/>
@ -828,7 +839,7 @@ the value and meanings for Primaries are adopted from Table 2 of ISO/IEC 23091-4
<enum value="0" label="reserved"/>
<enum value="1" label="ITU-R BT.709"/>
<enum value="2" label="unspecified"/>
<enum value="3" label="reserved"/>
<enum value="3" label="reserved2"/>
<enum value="4" label="ITU-R BT.470M"/>
<enum value="5" label="ITU-R BT.470BG - BT.601 625"/>
<enum value="6" label="ITU-R BT.601 525 - SMPTE 170M"/>
@ -865,49 +876,49 @@ in candelas per square meter (cd/m^2^).</documentation>
<extension type="stream copy" keep="1"/>
</element>
<element name="PrimaryRChromaticityX" path="\Segment\Tracks\TrackEntry\Video\Colour\MasteringMetadata\PrimaryRChromaticityX" id="0x55D1" type="float" minver="4" range="0-1" maxOccurs="1">
<documentation lang="en" purpose="definition">Red X chromaticity coordinate, as defined by CIE 1931.</documentation>
<documentation lang="en" purpose="definition">Red X chromaticity coordinate, as defined by [@!CIE-1931].</documentation>
<extension type="webmproject.org" webm="1"/>
<extension type="libmatroska" cppname="VideoRChromaX"/>
<extension type="stream copy" keep="1"/>
</element>
<element name="PrimaryRChromaticityY" path="\Segment\Tracks\TrackEntry\Video\Colour\MasteringMetadata\PrimaryRChromaticityY" id="0x55D2" type="float" minver="4" range="0-1" maxOccurs="1">
<documentation lang="en" purpose="definition">Red Y chromaticity coordinate, as defined by CIE 1931.</documentation>
<documentation lang="en" purpose="definition">Red Y chromaticity coordinate, as defined by [@!CIE-1931].</documentation>
<extension type="webmproject.org" webm="1"/>
<extension type="libmatroska" cppname="VideoRChromaY"/>
<extension type="stream copy" keep="1"/>
</element>
<element name="PrimaryGChromaticityX" path="\Segment\Tracks\TrackEntry\Video\Colour\MasteringMetadata\PrimaryGChromaticityX" id="0x55D3" type="float" minver="4" range="0-1" maxOccurs="1">
<documentation lang="en" purpose="definition">Green X chromaticity coordinate, as defined by CIE 1931.</documentation>
<documentation lang="en" purpose="definition">Green X chromaticity coordinate, as defined by [@!CIE-1931].</documentation>
<extension type="webmproject.org" webm="1"/>
<extension type="libmatroska" cppname="VideoGChromaX"/>
<extension type="stream copy" keep="1"/>
</element>
<element name="PrimaryGChromaticityY" path="\Segment\Tracks\TrackEntry\Video\Colour\MasteringMetadata\PrimaryGChromaticityY" id="0x55D4" type="float" minver="4" range="0-1" maxOccurs="1">
<documentation lang="en" purpose="definition">Green Y chromaticity coordinate, as defined by CIE 1931.</documentation>
<documentation lang="en" purpose="definition">Green Y chromaticity coordinate, as defined by [@!CIE-1931].</documentation>
<extension type="webmproject.org" webm="1"/>
<extension type="libmatroska" cppname="VideoGChromaY"/>
<extension type="stream copy" keep="1"/>
</element>
<element name="PrimaryBChromaticityX" path="\Segment\Tracks\TrackEntry\Video\Colour\MasteringMetadata\PrimaryBChromaticityX" id="0x55D5" type="float" minver="4" range="0-1" maxOccurs="1">
<documentation lang="en" purpose="definition">Blue X chromaticity coordinate, as defined by CIE 1931.</documentation>
<documentation lang="en" purpose="definition">Blue X chromaticity coordinate, as defined by [@!CIE-1931].</documentation>
<extension type="webmproject.org" webm="1"/>
<extension type="libmatroska" cppname="VideoBChromaX"/>
<extension type="stream copy" keep="1"/>
</element>
<element name="PrimaryBChromaticityY" path="\Segment\Tracks\TrackEntry\Video\Colour\MasteringMetadata\PrimaryBChromaticityY" id="0x55D6" type="float" minver="4" range="0-1" maxOccurs="1">
<documentation lang="en" purpose="definition">Blue Y chromaticity coordinate, as defined by CIE 1931.</documentation>
<documentation lang="en" purpose="definition">Blue Y chromaticity coordinate, as defined by [@!CIE-1931].</documentation>
<extension type="webmproject.org" webm="1"/>
<extension type="libmatroska" cppname="VideoBChromaY"/>
<extension type="stream copy" keep="1"/>
</element>
<element name="WhitePointChromaticityX" path="\Segment\Tracks\TrackEntry\Video\Colour\MasteringMetadata\WhitePointChromaticityX" id="0x55D7" type="float" minver="4" range="0-1" maxOccurs="1">
<documentation lang="en" purpose="definition">White X chromaticity coordinate, as defined by CIE 1931.</documentation>
<documentation lang="en" purpose="definition">White X chromaticity coordinate, as defined by [@!CIE-1931].</documentation>
<extension type="webmproject.org" webm="1"/>
<extension type="libmatroska" cppname="VideoWhitePointChromaX"/>
<extension type="stream copy" keep="1"/>
</element>
<element name="WhitePointChromaticityY" path="\Segment\Tracks\TrackEntry\Video\Colour\MasteringMetadata\WhitePointChromaticityY" id="0x55D8" type="float" minver="4" range="0-1" maxOccurs="1">
<documentation lang="en" purpose="definition">White Y chromaticity coordinate, as defined by CIE 1931.</documentation>
<documentation lang="en" purpose="definition">White Y chromaticity coordinate, as defined by [@!CIE-1931].</documentation>
<extension type="webmproject.org" webm="1"/>
<extension type="libmatroska" cppname="VideoWhitePointChromaY"/>
<extension type="stream copy" keep="1"/>
@ -973,7 +984,7 @@ Setting `ProjectionPoseYaw` to 180 or -180 degrees, with the `ProjectionPoseRoll
<extension type="libmatroska" cppname="VideoProjectionPoseYaw"/>
<extension type="stream copy" keep="1"/>
</element>
<element name="ProjectionPosePitch" path="\Segment\Tracks\TrackEntry\Video\Projection\ProjectionPosePitch" id="0x7674" type="float" range="&gt;= -0x5Ap+0, &lt;= 0x5Ap+0" minver="4" default="0x0p+0" minOccurs="1" maxOccurs="1">
<element name="ProjectionPosePitch" path="\Segment\Tracks\TrackEntry\Video\Projection\ProjectionPosePitch" id="0x7674" type="float" minver="4" range="&gt;= -0x5Ap+0, &lt;= 0x5Ap+0" default="0x0p+0" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">Specifies a pitch rotation to the projection.
Value represents a counter-clockwise rotation, in degrees, around the right vector. This rotation must be applied
@ -983,7 +994,7 @@ The value of this element **MUST** be in the -90 to 90 degree range, both includ
<extension type="libmatroska" cppname="VideoProjectionPosePitch"/>
<extension type="stream copy" keep="1"/>
</element>
<element name="ProjectionPoseRoll" path="\Segment\Tracks\TrackEntry\Video\Projection\ProjectionPoseRoll" id="0x7675" type="float" range="&gt;= -0xB4p+0, &lt;= 0xB4p+0" minver="4" default="0x0p+0" minOccurs="1" maxOccurs="1">
<element name="ProjectionPoseRoll" path="\Segment\Tracks\TrackEntry\Video\Projection\ProjectionPoseRoll" id="0x7675" type="float" minver="4" range="&gt;= -0xB4p+0, &lt;= 0xB4p+0" default="0x0p+0" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">Specifies a roll rotation to the projection.
Value represents a counter-clockwise rotation, in degrees, around the forward vector. This rotation must be applied
@ -1030,6 +1041,47 @@ Setting `ProjectionPoseRoll` to 180 or -180 degrees, with the `ProjectionPoseYaw
<extension type="stream copy" keep="1"/>
<extension type="webmproject.org" webm="1"/>
</element>
<element name="Emphasis" path="\Segment\Tracks\TrackEntry\Audio\Emphasis" id="0x52F1" type="uinteger" minver="5" default="0" minOccurs="1" maxOccurs="1" >
<documentation lang="en" purpose="definition">Audio emphasis applied on audio samples. The player **MUST** apply the inverse emphasis to get the proper audio samples.</documentation>
<restriction>
<enum value="0" label="No emphasis"/>
<enum value="1" label="CD audio">
<documentation lang="en" purpose="definition">First order filter with zero point at 50 microseconds and a pole at 15 microseconds. Also found on DVD Audio and MPEG audio.</documentation>
</enum>
<enum value="2" label="reserved"/>
<enum value="3" label="CCIT J.17">
<documentation lang="en" purpose="definition">Defined in [@!ITU-J.17].</documentation>
</enum>
<enum value="4" label="FM 50">
<documentation lang="en" purpose="definition">FM Radio in Europe. RC Filter with a time constant of 50 microseconds.</documentation>
</enum>
<enum value="5" label="FM 75">
<documentation lang="en" purpose="definition">FM Radio in the USA. RC Filter with a time constant of 75 microseconds.</documentation>
</enum>
<enum value="10" label="Phono RIAA">
<documentation lang="en" purpose="definition">Phono filter with time constants of t1=3180, t2=318 and t3=75 microseconds. [@!NAB1964]</documentation>
</enum>
<enum value="11" label="Phono IEC N78">
<documentation lang="en" purpose="definition">Phono filter with time constants of t1=3180, t2=450 and t3=50 microseconds.</documentation>
</enum>
<enum value="12" label="Phono TELDEC">
<documentation lang="en" purpose="definition">Phono filter with time constants of t1=3180, t2=318 and t3=50 microseconds.</documentation>
</enum>
<enum value="13" label="Phono EMI">
<documentation lang="en" purpose="definition">Phono filter with time constants of t1=2500, t2=500 and t3=70 microseconds.</documentation>
</enum>
<enum value="14" label="Phono Columbia LP">
<documentation lang="en" purpose="definition">Phono filter with time constants of t1=1590, t2=318 and t3=100 microseconds.</documentation>
</enum>
<enum value="15" label="Phono LONDON">
<documentation lang="en" purpose="definition">Phono filter with time constants of t1=1590, t2=318 and t3=50 microseconds.</documentation>
</enum>
<enum value="16" label="Phono NARTB">
<documentation lang="en" purpose="definition">Phono filter with time constants of t1=3180, t2=318 and t3=100 microseconds.</documentation>
</enum>
</restriction>
<extension type="stream copy" keep="1"/>
</element>
<element name="TrackOperation" path="\Segment\Tracks\TrackEntry\TrackOperation" id="0xE2" type="master" minver="3" maxOccurs="1">
<documentation lang="en" purpose="definition">Operation that needs to be applied on tracks to create this virtual track.
For more details look at (#track-operation).</documentation>
@ -1137,23 +1189,24 @@ Each block **MUST** be decompressable even if no previous block is available in
</element>
<element name="ContentCompAlgo" path="\Segment\Tracks\TrackEntry\ContentEncodings\ContentEncoding\ContentCompression\ContentCompAlgo" id="0x4254" type="uinteger" default="0" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">The compression algorithm used.</documentation>
<documentation lang="en" purpose="usage notes">Compression method "1" (bzlib) and "2" (lzo1x) are lacking proper documentation on the format which limits implementation possibilities.
Due to licensing conflicts on commonly available libraries compression methods "2" (lzo1x) does not offer widespread interoperability.
A Matroska Writer **SHOULD NOT** use these compression methods by default.
A Matroska Reader **MAY** support methods "1" and "2" as possible, and **SHOULD** support other methods.</documentation>
<restriction>
<enum value="0" label="zlib">
<documentation lang="en" purpose="definition">zlib compression [@!RFC1950].</documentation>
</enum>
<enum value="1" label="bzlib">
<documentation lang="en" purpose="definition">bzip2 compression [@!BZIP2], **SHOULD NOT** be used; see usage notes.</documentation>
<documentation lang="en" purpose="definition">bzip2 compression [@?BZIP2], **SHOULD NOT** be used; see usage notes.</documentation>
</enum>
<enum value="2" label="lzo1x">
<documentation lang="en" purpose="definition">Lempel-Ziv-Oberhumer compression [@!LZO], **SHOULD NOT** be used; see usage notes.</documentation>
<documentation lang="en" purpose="definition">Lempel-Ziv-Oberhumer compression [@?LZO], **SHOULD NOT** be used; see usage notes.</documentation>
</enum>
<enum value="3" label="Header Stripping">
<documentation lang="en" purpose="definition">Octets in `ContentCompSettings` ((#contentcompsettings-element)) have been stripped from each frame.</documentation>
</enum>
</restriction>
<documentation lang="en" purpose="usage notes">Compression method "1" (bzlib) and "2" (lzo1x) are lacking proper documentation on the format which limits implementation possibilities.
Due to licensing conflicts on commonly available libraries compression methods "2" (lzo1x) does not offer widespread interoperability.
Decoding implementations **MAY** support methods "1" and "2" as possible. The use of these compression methods **SHOULD NOT** be used as a default.</documentation>
<extension type="stream copy" keep="1"/>
</element>
<element name="ContentCompSettings" path="\Segment\Tracks\TrackEntry\ContentEncodings\ContentEncoding\ContentCompression\ContentCompSettings" id="0x4255" type="binary" maxOccurs="1">
@ -1163,29 +1216,31 @@ the bytes that were removed from the beginning of each frames of the track.</doc
</element>
<element name="ContentEncryption" path="\Segment\Tracks\TrackEntry\ContentEncodings\ContentEncoding\ContentEncryption" id="0x5035" type="master" maxOccurs="1">
<documentation lang="en" purpose="definition">Settings describing the encryption used.
This Element **MUST** be present if the value of `ContentEncodingType` is 1 (encryption) and **MUST** be ignored otherwise.</documentation>
This Element **MUST** be present if the value of `ContentEncodingType` is 1 (encryption) and **MUST** be ignored otherwise.
A Matroska Player **MAY** support encryption.</documentation>
<extension type="webmproject.org" webm="1"/>
<extension type="stream copy" keep="1"/>
</element>
<element name="ContentEncAlgo" path="\Segment\Tracks\TrackEntry\ContentEncodings\ContentEncoding\ContentEncryption\ContentEncAlgo" id="0x47E1" type="uinteger" default="0" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">The encryption algorithm used.
The value "0" means that the contents have not been encrypted.</documentation>
<documentation lang="en" purpose="definition">The encryption algorithm used.</documentation>
<restriction>
<enum value="0" label="Not encrypted"/>
<enum value="0" label="Not encrypted">
<documentation lang="en" purpose="definition">The data are not encrypted.</documentation>
</enum>
<enum value="1" label="DES">
<documentation lang="en" purpose="definition">Data Encryption Standard (DES) [@!FIPS.46-3].</documentation>
<documentation lang="en" purpose="definition">Data Encryption Standard (DES) [@?FIPS.46-3].</documentation>
</enum>
<enum value="2" label="3DES">
<documentation lang="en" purpose="definition">Triple Data Encryption Algorithm [@!SP.800-67].</documentation>
<documentation lang="en" purpose="definition">Triple Data Encryption Algorithm [@?SP.800-67].</documentation>
</enum>
<enum value="3" label="Twofish">
<documentation lang="en" purpose="definition">Twofish Encryption Algorithm [@!Twofish].</documentation>
<documentation lang="en" purpose="definition">Twofish Encryption Algorithm [@?Twofish].</documentation>
</enum>
<enum value="4" label="Blowfish">
<documentation lang="en" purpose="definition">Blowfish Encryption Algorithm [@!Blowfish].</documentation>
<documentation lang="en" purpose="definition">Blowfish Encryption Algorithm [@?Blowfish].</documentation>
</enum>
<enum value="5" label="AES">
<documentation lang="en" purpose="definition">Advanced Encryption Standard (AES) [@!FIPS.197].</documentation>
<documentation lang="en" purpose="definition">Advanced Encryption Standard (AES) [@?FIPS.197].</documentation>
</enum>
</restriction>
<extension type="webmproject.org" webm="1"/>
@ -1197,20 +1252,20 @@ The value "0" means that the contents have not been encrypted.</documentation>
<extension type="stream copy" keep="1"/>
</element>
<element name="ContentEncAESSettings" path="\Segment\Tracks\TrackEntry\ContentEncodings\ContentEncoding\ContentEncryption\ContentEncAESSettings" id="0x47E7" type="master" minver="4" maxOccurs="1">
<documentation lang="en" purpose="definition">Settings describing the encryption algorithm used.
It **MUST** be ignored if `ContentEncAlgo` is not AES (5).</documentation>
<documentation lang="en" purpose="definition">Settings describing the encryption algorithm used.</documentation>
<implementation_note note_attribute="maxOccurs">ContentEncAESSettings **MUST NOT** be set (maxOccurs=0) if ContentEncAlgo is not AES (5).</implementation_note>
<extension type="webmproject.org" webm="1"/>
<extension type="stream copy" keep="1"/>
</element>
<element name="AESSettingsCipherMode" path="\Segment\Tracks\TrackEntry\ContentEncodings\ContentEncoding\ContentEncryption\ContentEncAESSettings\AESSettingsCipherMode" id="0x47E8" type="uinteger" minver="4" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">The AES cipher mode used in the encryption.
It **MUST** be ignored if `ContentEncAlgo` is not AES (5).</documentation>
<documentation lang="en" purpose="definition">The AES cipher mode used in the encryption.</documentation>
<implementation_note note_attribute="maxOccurs">AESSettingsCipherMode **MUST NOT** be set (maxOccurs=0) if ContentEncAlgo is not AES (5).</implementation_note>
<restriction>
<enum value="1" label="AES-CTR">
<documentation lang="en" purpose="definition">Counter [@!SP.800-38A].</documentation>
<documentation lang="en" purpose="definition">Counter [@?SP.800-38A].</documentation>
</enum>
<enum value="2" label="AES-CBC">
<documentation lang="en" purpose="definition">Cipher Block Chaining [@!SP.800-38A].</documentation>
<documentation lang="en" purpose="definition">Cipher Block Chaining [@?SP.800-38A].</documentation>
</enum>
</restriction>
<extension type="webmproject.org" webm="1"/>
@ -1260,7 +1315,7 @@ All entries are local to the Segment.</documentation>
<extension type="webmproject.org" webm="1"/>
</element>
<element name="CueClusterPosition" path="\Segment\Cues\CuePoint\CueTrackPositions\CueClusterPosition" id="0xF1" type="uinteger" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">The Segment Position of the Cluster containing the associated Block.</documentation>
<documentation lang="en" purpose="definition">The Segment Position ((#segment-position)) of the Cluster containing the associated Block.</documentation>
<extension type="webmproject.org" webm="1"/>
</element>
<element name="CueRelativePosition" path="\Segment\Cues\CuePoint\CueTrackPositions\CueRelativePosition" id="0xF0" type="uinteger" minver="4" maxOccurs="1">
@ -1278,7 +1333,7 @@ If missing, the track's DefaultDuration does not apply and no duration informati
<extension type="webmproject.org" webm="1"/>
</element>
<element name="CueCodecState" path="\Segment\Cues\CuePoint\CueTrackPositions\CueCodecState" id="0xEA" type="uinteger" minver="2" default="0" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">The Segment Position of the Codec State corresponding to this Cue Element.
<documentation lang="en" purpose="definition">The Segment Position ((#segment-position)) of the Codec State corresponding to this Cue Element.
0 means that the data is taken from the initial Track Entry.</documentation>
</element>
<element name="CueReference" path="\Segment\Cues\CuePoint\CueTrackPositions\CueReference" id="0xDB" type="master" minver="2">
@ -1310,8 +1365,8 @@ If missing, the track's DefaultDuration does not apply and no duration informati
<element name="FileName" path="\Segment\Attachments\AttachedFile\FileName" id="0x466E" type="utf-8" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">Filename of the attached file.</documentation>
</element>
<element name="FileMimeType" path="\Segment\Attachments\AttachedFile\FileMimeType" id="0x4660" type="string" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">MIME type of the file.</documentation>
<element name="FileMediaType" path="\Segment\Attachments\AttachedFile\FileMediaType" id="0x4660" type="string" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">Media type of the file following the [@!RFC6838] format.</documentation>
<extension type="libmatroska" cppname="MimeType"/>
<extension type="stream copy" keep="1"/>
</element>
@ -1360,6 +1415,16 @@ For more detailed information, look at the Chapters explanation in (#chapters).<
<element name="EditionFlagOrdered" path="\Segment\Chapters\EditionEntry\EditionFlagOrdered" id="0x45DD" type="uinteger" range="0-1" default="0" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">Set to 1 if the chapters can be defined multiple times and the order to play them is enforced; see (#editionflagordered).</documentation>
</element>
<element name="EditionDisplay" path="\Segment\Chapters\EditionEntry\EditionDisplay" id="0x4520" type="master" minver="5">
<documentation lang="en" purpose="definition">Contains a possible string to use for the edition display for the given languages.</documentation>
</element>
<element name="EditionString" path="\Segment\Chapters\EditionEntry\EditionDisplay\EditionString" id="0x4521" type="utf-8" minver="5" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">Contains the string to use as the edition name.</documentation>
</element>
<element name="EditionLanguageIETF" path="\Segment\Chapters\EditionEntry\EditionDisplay\EditionLanguageIETF" id="0x45E4" type="string" minver="5">
<documentation lang="en" purpose="definition">One language corresponding to the EditionString,
in the [@!BCP47] form; see (#language-codes) on language codes.</documentation>
</element>
<element name="ChapterAtom" path="\Segment\Chapters\EditionEntry\+ChapterAtom" id="0xB6" type="master" minOccurs="1" recursive="1">
<documentation lang="en" purpose="definition">Contains the atom information to use as the chapter atom (apply to all tracks).</documentation>
<extension type="webmproject.org" webm="1"/>
@ -1371,7 +1436,7 @@ For more detailed information, look at the Chapters explanation in (#chapters).<
</element>
<element name="ChapterStringUID" path="\Segment\Chapters\EditionEntry\+ChapterAtom\ChapterStringUID" id="0x5654" type="utf-8" minver="3" maxOccurs="1">
<documentation lang="en" purpose="definition">A unique string ID to identify the Chapter.
Use for WebVTT cue identifier storage [@!WebVTT].</documentation>
Use for WebVTT cue identifier storage [@?WebVTT].</documentation>
<extension type="webmproject.org" webm="1"/>
</element>
<element name="ChapterTimeStart" path="\Segment\Chapters\EditionEntry\+ChapterAtom\ChapterTimeStart" id="0x91" type="uinteger" minOccurs="1" maxOccurs="1">
@ -1395,13 +1460,45 @@ the last frame it includes, especially for the `ChapterAtom` using the last fram
When disabled, the movie **SHOULD** skip all the content between the TimeStart and TimeEnd of this chapter; see (#chapter-flags) on Chapter flags.</documentation>
<extension type="other document" spec="control-track"/>
</element>
<element name="ChapterSegmentUID" path="\Segment\Chapters\EditionEntry\+ChapterAtom\ChapterSegmentUID" id="0x6E67" type="binary" range="&gt;0" length="16" maxOccurs="1">
<documentation lang="en" purpose="definition">The SegmentUID of another Segment to play during this chapter.</documentation>
<documentation lang="en" purpose="usage notes">The value **MUST NOT** be the `SegmentUID` value of the `Segment` it belongs to.</documentation>
<implementation_note note_attribute="minOccurs">ChapterSegmentUID **MUST** be set (minOccurs=1) if ChapterSegmentEditionUID is used; see (#medium-linking) on medium-linking Segments.</implementation_note>
<element name="ChapterSegmentUUID" path="\Segment\Chapters\EditionEntry\+ChapterAtom\ChapterSegmentUUID" id="0x6E67" type="binary" range="&gt;0" length="16" maxOccurs="1">
<documentation lang="en" purpose="definition">The SegmentUUID of another Segment to play during this chapter (128 bits).
Like the SegmentUUID, it is a Universally Unique IDentifier stored in binary form [@!RFC4122].</documentation>
<documentation lang="en" purpose="usage notes">The value **MUST NOT** be the `SegmentUUID` value of the `Segment` it belongs to.</documentation>
<implementation_note note_attribute="minOccurs">ChapterSegmentUUID **MUST** be set (minOccurs=1) if ChapterSegmentEditionUID is used; see (#medium-linking) on medium-linking Segments.</implementation_note>
<extension type="libmatroska" cppname="ChapterSegmentUID"/>
</element>
<element name="ChapterSkipType" path="\Segment\Chapters\EditionEntry\+ChapterAtom\ChapterSkipType" id="0x4588" type="uinteger" maxOccurs="1" minver="5">
<documentation lang="en" purpose="definition">Indicate what type of content the ChapterAtom contains and might be skipped. It can be used to automatically skip content based on the type.
If a `ChapterAtom` is inside a `ChapterAtom` that has a `ChapterSkipType` set, it **MUST NOT** have a `ChapterSkipType` or have a `ChapterSkipType` with the same value as it's parent `ChapterAtom`.
If the `ChapterAtom` doesn't contain a `ChapterTimeEnd`, the value of the `ChapterSkipType` is only valid until the next `ChapterAtom` with a `ChapterSkipType` value or the end of the file.
</documentation>
<extension type="webmproject.org" webm="0"/>
<restriction>
<enum value="0" label="No Skipping">
<documentation lang="en" purpose="definition">Content which should not be skipped.</documentation>
</enum>
<enum value="1" label="Opening Credits">
<documentation lang="en" purpose="definition">Credits usually found at the beginning of the content.</documentation>
</enum>
<enum value="2" label="End Credits">
<documentation lang="en" purpose="definition">Credits usually found at the end of the content.</documentation>
</enum>
<enum value="3" label="Recap">
<documentation lang="en" purpose="definition">Recap of previous episodes of the content, usually found around the beginning.</documentation>
</enum>
<enum value="4" label="Next Preview">
<documentation lang="en" purpose="definition">Preview of the next episode of the content, usually found around the end. It may contain spoilers the user wants to avoid.</documentation>
</enum>
<enum value="5" label="Preview">
<documentation lang="en" purpose="definition">Preview of the current episode of the content, usually found around the beginning. It may contain spoilers the user want to avoid.</documentation>
</enum>
<enum value="6" label="Advertisement">
<documentation lang="en" purpose="definition">Advertisement within the content.</documentation>
</enum>
</restriction>
</element>
<element name="ChapterSegmentEditionUID" path="\Segment\Chapters\EditionEntry\+ChapterAtom\ChapterSegmentEditionUID" id="0x6EBC" type="uinteger" range="not 0" maxOccurs="1">
<documentation lang="en" purpose="definition">The EditionUID to play from the Segment linked in ChapterSegmentUID.
<documentation lang="en" purpose="definition">The EditionUID to play from the Segment linked in ChapterSegmentUUID.
If ChapterSegmentEditionUID is undeclared, then no Edition of the linked Segment is used; see (#medium-linking) on medium-linking Segments.</documentation>
</element>
<element name="ChapterPhysicalEquiv" path="\Segment\Chapters\EditionEntry\+ChapterAtom\ChapterPhysicalEquiv" id="0x63C3" type="uinteger" maxOccurs="1">
@ -1430,19 +1527,21 @@ Absence of this Element indicates that the Chapter **SHOULD** be applied to any
</element>
<element name="ChapLanguage" path="\Segment\Chapters\EditionEntry\+ChapterAtom\ChapterDisplay\ChapLanguage" id="0x437C" type="string" default="eng" minOccurs="1">
<documentation lang="en" purpose="definition">A language corresponding to the string,
in the bibliographic ISO-639-2 form [@!ISO639-2].
This Element **MUST** be ignored if a ChapLanguageIETF Element is used within the same ChapterDisplay Element.</documentation>
in the Matroska languages form; see (#language-codes) on language codes.
This Element **MUST** be ignored if a ChapLanguageBCP47 Element is used within the same ChapterDisplay Element.</documentation>
<extension type="webmproject.org" webm="1"/>
<extension type="libmatroska" cppname="ChapterLanguage"/>
</element>
<element name="ChapLanguageIETF" path="\Segment\Chapters\EditionEntry\+ChapterAtom\ChapterDisplay\ChapLanguageIETF" id="0x437D" type="string" minver="4">
<documentation lang="en" purpose="definition">Specifies a language corresponding to the ChapString in the format defined in [@!BCP47]
and using the IANA Language Subtag Registry [@!IANALangRegistry].
If a ChapLanguageIETF Element is used, then any ChapLanguage and ChapCountry Elements used in the same ChapterDisplay **MUST** be ignored.</documentation>
<element name="ChapLanguageBCP47" path="\Segment\Chapters\EditionEntry\+ChapterAtom\ChapterDisplay\ChapLanguageBCP47" id="0x437D" type="string" minver="4">
<documentation lang="en" purpose="definition">A language corresponding to the ChapString,
in the [@!BCP47] form; see (#language-codes) on language codes.
If a ChapLanguageBCP47 Element is used, then any ChapLanguage and ChapCountry Elements used in the same ChapterDisplay **MUST** be ignored.</documentation>
<extension type="libmatroska" cppname="ChapLanguageIETF"/>
</element>
<element name="ChapCountry" path="\Segment\Chapters\EditionEntry\+ChapterAtom\ChapterDisplay\ChapCountry" id="0x437E" type="string">
<documentation lang="en" purpose="definition">A country corresponding to the string, using the same 2 octets country-codes as in Internet domains [@!IANADomains] based on [@!ISO3166-1] alpha-2 codes.
This Element **MUST** be ignored if a ChapLanguageIETF Element is used within the same ChapterDisplay Element.</documentation>
<documentation lang="en" purpose="definition">A country corresponding to the string,
in the Matroska countries form; see (#country-codes) on country codes.
This Element **MUST** be ignored if a ChapLanguageBCP47 Element is used within the same ChapterDisplay Element.</documentation>
<extension type="webmproject.org" webm="1"/>
<extension type="libmatroska" cppname="ChapterCountry"/>
</element>
@ -1482,7 +1581,7 @@ the data correspond to the binary DVD cell pre/post commands; see (#menu-feature
</element>
<element name="Tags" path="\Segment\Tags" id="0x1254C367" type="master">
<documentation lang="en" purpose="definition">Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole.
A list of valid tags can be found in [@!MatroskaTags].</documentation>
A list of valid tags can be found in [@?MatroskaTags].</documentation>
<extension type="webmproject.org" webm="1"/>
</element>
<element name="Tag" path="\Segment\Tags\Tag" id="0x7373" type="master" minOccurs="1">
@ -1525,30 +1624,30 @@ If empty or not present, then the Tag describes everything in the Segment.</docu
</element>
<element name="TargetType" path="\Segment\Tags\Tag\Targets\TargetType" id="0x63CA" type="string" maxOccurs="1">
<documentation lang="en" purpose="definition">An informational string that can be used to display the logical level of the target like "ALBUM", "TRACK", "MOVIE", "CHAPTER", etc
; see Section 6.4 of [@!MatroskaTags].</documentation>
; see Section 6.4 of [@?MatroskaTags].</documentation>
<restriction>
<enum value="COLLECTION" label="COLLECTION"/>
<enum value="EDITION" label="EDITION"/>
<enum value="ISSUE" label="ISSUE"/>
<enum value="VOLUME" label="VOLUME"/>
<enum value="OPUS" label="OPUS"/>
<enum value="SEASON" label="SEASON"/>
<enum value="SEQUEL" label="SEQUEL"/>
<enum value="ALBUM" label="ALBUM"/>
<enum value="OPERA" label="OPERA"/>
<enum value="CONCERT" label="CONCERT"/>
<enum value="MOVIE" label="MOVIE"/>
<enum value="EPISODE" label="EPISODE"/>
<enum value="PART" label="PART"/>
<enum value="SESSION" label="SESSION"/>
<enum value="TRACK" label="TRACK"/>
<enum value="SONG" label="SONG"/>
<enum value="CHAPTER" label="CHAPTER"/>
<enum value="SUBTRACK" label="SUBTRACK"/>
<enum value="PART" label="PART"/>
<enum value="MOVEMENT" label="MOVEMENT"/>
<enum value="SCENE" label="SCENE"/>
<enum value="SHOT" label="SHOT"/>
<enum value="COLLECTION" label="TargetTypeValue 70"/>
<enum value="EDITION" label="TargetTypeValue 60"/>
<enum value="ISSUE" label="TargetTypeValue 60"/>
<enum value="VOLUME" label="TargetTypeValue 60"/>
<enum value="OPUS" label="TargetTypeValue 60"/>
<enum value="SEASON" label="TargetTypeValue 60"/>
<enum value="SEQUEL" label="TargetTypeValue 60"/>
<enum value="ALBUM" label="TargetTypeValue 50"/>
<enum value="OPERA" label="TargetTypeValue 50"/>
<enum value="CONCERT" label="TargetTypeValue 50"/>
<enum value="MOVIE" label="TargetTypeValue 50"/>
<enum value="EPISODE" label="TargetTypeValue 50"/>
<enum value="PART" label="TargetTypeValue 40"/>
<enum value="SESSION" label="TargetTypeValue 40"/>
<enum value="TRACK" label="TargetTypeValue 30"/>
<enum value="SONG" label="TargetTypeValue 30"/>
<enum value="CHAPTER" label="TargetTypeValue 30"/>
<enum value="SUBTRACK" label="TargetTypeValue 20"/>
<enum value="PART" label="TargetTypeValue 20"/>
<enum value="MOVEMENT" label="TargetTypeValue 20"/>
<enum value="SCENE" label="TargetTypeValue 20"/>
<enum value="SHOT" label="TargetTypeValue 10"/>
</restriction>
<extension type="webmproject.org" webm="1"/>
<extension type="libmatroska" cppname="TagTargetType"/>
@ -1584,16 +1683,17 @@ If set to any other value, it **MUST** match the `FileUID` value of an attachmen
<extension type="webmproject.org" webm="1"/>
</element>
<element name="TagLanguage" path="\Segment\Tags\Tag\+SimpleTag\TagLanguage" id="0x447A" type="string" default="und" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">Specifies the language of the tag specified, in the Matroska languages form;
see (#language-codes) on language codes.
This Element **MUST** be ignored if the TagLanguageIETF Element is used within the same SimpleTag Element.</documentation>
<documentation lang="en" purpose="definition">Specifies the language of the tag specified,
in the Matroska languages form; see (#language-codes) on language codes.
This Element **MUST** be ignored if the TagLanguageBCP47 Element is used within the same SimpleTag Element.</documentation>
<extension type="webmproject.org" webm="1"/>
<extension type="libmatroska" cppname="TagLangue"/>
</element>
<element name="TagLanguageIETF" path="\Segment\Tags\Tag\+SimpleTag\TagLanguageIETF" id="0x447B" type="string" minver="4" maxOccurs="1">
<documentation lang="en" purpose="definition">Specifies the language used in the TagString according to [@!BCP47]
and using the IANA Language Subtag Registry [@!IANALangRegistry].
<element name="TagLanguageBCP47" path="\Segment\Tags\Tag\+SimpleTag\TagLanguageBCP47" id="0x447B" type="string" minver="4" maxOccurs="1">
<documentation lang="en" purpose="definition">The language used in the TagString,
in the [@!BCP47] form; see (#language-codes) on language codes.
If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.</documentation>
<extension type="libmatroska" cppname="TagLanguageIETF"/>
</element>
<element name="TagDefault" path="\Segment\Tags\Tag\+SimpleTag\TagDefault" id="0x4484" type="uinteger" range="0-1" default="1" minOccurs="1" maxOccurs="1">
<documentation lang="en" purpose="definition">A boolean value to indicate if this is the default/original language to use for the given tag.</documentation>

320
format/matroska/ebml_matroska/ebml_matroska_gen.go
View File

@ -22,11 +22,11 @@ const (
SeekIDID = 0x53ab
SeekPositionID = 0x53ac
InfoID = 0x1549a966
SegmentUIDID = 0x73a4
SegmentUUIDID = 0x73a4
SegmentFilenameID = 0x7384
PrevUIDID = 0x3cb923
PrevUUIDID = 0x3cb923
PrevFilenameID = 0x3c83ab
NextUIDID = 0x3eb923
NextUUIDID = 0x3eb923
NextFilenameID = 0x3e83bb
SegmentFamilyID = 0x4444
ChapterTranslateID = 0x6924
@ -51,8 +51,8 @@ const (
BlockVirtualID = 0xa2
BlockAdditionsID = 0x75a1
BlockMoreID = 0xa6
BlockAddIDID = 0xee
BlockAdditionalID = 0xa5
BlockAddIDID = 0xee
BlockDurationID = 0x9b
ReferencePriorityID = 0xfa
ReferenceBlockID = 0xfb
@ -98,7 +98,7 @@ const (
BlockAddIDExtraDataID = 0x41ed
NameID = 0x536e
LanguageID = 0x22b59c
LanguageIETFID = 0x22b59d
LanguageBCP47ID = 0x22b59d
CodecIDID = 0x86
CodecPrivateID = 0x63a2
CodecNameID = 0x258688
@ -170,6 +170,7 @@ const (
ChannelsID = 0x9f
ChannelPositionsID = 0x7d7b
BitDepthID = 0x6264
EmphasisID = 0x52f1
TrackOperationID = 0xe2
TrackCombinePlanesID = 0xe3
TrackPlaneID = 0xe4
@ -218,7 +219,7 @@ const (
AttachedFileID = 0x61a7
FileDescriptionID = 0x467e
FileNameID = 0x466e
FileMimeTypeID = 0x4660
FileMediaTypeID = 0x4660
FileDataID = 0x465c
FileUIDID = 0x46ae
FileReferralID = 0x4675
@ -230,6 +231,9 @@ const (
EditionFlagHiddenID = 0x45bd
EditionFlagDefaultID = 0x45db
EditionFlagOrderedID = 0x45dd
EditionDisplayID = 0x4520
EditionStringID = 0x4521
EditionLanguageIETFID = 0x45e4
ChapterAtomID = 0xb6
ChapterUIDID = 0x73c4
ChapterStringUIDID = 0x5654
@ -237,7 +241,8 @@ const (
ChapterTimeEndID = 0x92
ChapterFlagHiddenID = 0x98
ChapterFlagEnabledID = 0x4598
ChapterSegmentUIDID = 0x6e67
ChapterSegmentUUIDID = 0x6e67
ChapterSkipTypeID = 0x4588
ChapterSegmentEditionUIDID = 0x6ebc
ChapterPhysicalEquivID = 0x63c3
ChapterTrackID = 0x8f
@ -245,7 +250,7 @@ const (
ChapterDisplayID = 0x80
ChapStringID = 0x85
ChapLanguageID = 0x437c
ChapLanguageIETFID = 0x437d
ChapLanguageBCP47ID = 0x437d
ChapCountryID = 0x437e
ChapProcessID = 0x6944
ChapProcessCodecIDID = 0x6955
@ -265,7 +270,7 @@ const (
SimpleTagID = 0x67c8
TagNameID = 0x45a3
TagLanguageID = 0x447a
TagLanguageIETFID = 0x447b
TagLanguageBCP47ID = 0x447b
TagDefaultID = 0x4484
TagDefaultBogusID = 0x44b4
TagStringID = 0x4487
@ -275,7 +280,7 @@ const (
var Segment = ebml.Tag{
SeekHeadID: {
Name: "seek_head",
Definition: "Contains the Segment Position of other Top-Level Elements.",
Definition: "Contains seeking information of Top-Level Elements; see (#data-layout).",
Type: ebml.Master, Tag: SeekHead,
},
InfoID: {
@ -310,7 +315,7 @@ var Segment = ebml.Tag{
},
TagsID: {
Name: "tags",
Definition: "Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@!MatroskaTags].",
Definition: "Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@?MatroskaTags].",
Type: ebml.Master, Tag: Tags,
},
}
@ -326,20 +331,20 @@ var SeekHead = ebml.Tag{
var Seek = ebml.Tag{
SeekIDID: {
Name: "seek_id",
Definition: "The binary ID corresponding to the Element name.",
Definition: "The binary EBML ID of a Top-Level Element.",
Type: ebml.Binary,
},
SeekPositionID: {
Name: "seek_position",
Definition: "The Segment Position of the Element.",
Definition: "The Segment Position ((#segment-position)) of a Top-Level Element.",
Type: ebml.Uinteger,
},
}
var Info = ebml.Tag{
SegmentUIDID: {
Name: "segment_uid",
Definition: "A randomly generated unique ID to identify the Segment amongst many others (128 bits).",
SegmentUUIDID: {
Name: "segment_uuid",
Definition: "A randomly generated unique ID to identify the Segment amongst many others (128 bits). It is effectively a Universally Unique IDentifier stored in binary form [@!RFC4122].",
Type: ebml.Binary,
},
SegmentFilenameID: {
@ -347,9 +352,9 @@ var Info = ebml.Tag{
Definition: "A filename corresponding to this Segment.",
Type: ebml.UTF8,
},
PrevUIDID: {
Name: "prev_uid",
Definition: "A unique ID to identify the previous Segment of a Linked Segment (128 bits).",
PrevUUIDID: {
Name: "prev_uuid",
Definition: "A unique ID to identify the previous Segment of a Linked Segment (128 bits). Like the SegmentUUID, it is a Universally Unique IDentifier stored in binary form [@!RFC4122].",
Type: ebml.Binary,
},
PrevFilenameID: {
@ -357,9 +362,9 @@ var Info = ebml.Tag{
Definition: "A filename corresponding to the file of the previous Linked Segment.",
Type: ebml.UTF8,
},
NextUIDID: {
Name: "next_uid",
Definition: "A unique ID to identify the next Segment of a Linked Segment (128 bits).",
NextUUIDID: {
Name: "next_uuid",
Definition: "A unique ID to identify the next Segment of a Linked Segment (128 bits). Like the SegmentUUID, it is a Universally Unique IDentifier stored in binary form [@!RFC4122].",
Type: ebml.Binary,
},
NextFilenameID: {
@ -369,7 +374,7 @@ var Info = ebml.Tag{
},
SegmentFamilyID: {
Name: "segment_family",
Definition: "A randomly generated unique ID that all Segments of a Linked Segment **MUST** share (128 bits).",
Definition: "A randomly generated unique ID that all Segments of a Linked Segment **MUST** share (128 bits). It is effectively a Universally Unique IDentifier stored in binary form [@!RFC4122].",
Type: ebml.Binary,
},
ChapterTranslateID: {
@ -496,7 +501,7 @@ var BlockGroup = ebml.Tag{
},
BlockAdditionsID: {
Name: "block_additions",
Definition: "Contain additional blocks to complete the main one. An EBML parser that has no knowledge of the Block structure could still see and use/skip these data.",
Definition: "Contain additional binary data to complete the main one; see Codec BlockAdditions section of [@?MatroskaCodec] for more information. An EBML parser that has no knowledge of the Block structure could still see and use/skip these data.",
Type: ebml.Master, Tag: BlockAdditions,
},
BlockDurationID: {
@ -550,16 +555,16 @@ var BlockAdditions = ebml.Tag{
}
var BlockMore = ebml.Tag{
BlockAddIDID: {
Name: "block_add_id",
Definition: "An ID to identify the BlockAdditional level. If BlockAddIDType of the corresponding block is 0, this value is also the value of BlockAddIDType for the meaning of the content of BlockAdditional.",
Type: ebml.Uinteger,
},
BlockAdditionalID: {
Name: "block_additional",
Definition: "Interpreted by the codec as it wishes (using the BlockAddID).",
Type: ebml.Binary,
},
BlockAddIDID: {
Name: "block_add_id",
Definition: "An ID to identify how to interpret the BlockAdditional data; see Codec BlockAdditions section of [@?MatroskaCodec] for more information. A value of 1 indicates that the meaning of the BlockAdditional data is defined by the codec. Any other value indicates the meaning of the BlockAdditional data is found in the BlockAddIDType found in the TrackEntry.",
Type: ebml.Uinteger,
},
}
var Slices = ebml.Tag{
@ -736,7 +741,7 @@ var TrackEntry = ebml.Tag{
},
TrackTimestampScaleID: {
Name: "track_timestamp_scale",
Definition: "DEPRECATED, DO NOT USE. The scale to apply on this track to work at normal speed in relation with other tracks (mostly used to adjust video speed when the audio length differs).",
Definition: "The scale to apply on this track to work at normal speed in relation with other tracks (mostly used to adjust video speed when the audio length differs).",
Type: ebml.Float,
},
TrackOffsetID: {
@ -761,17 +766,17 @@ var TrackEntry = ebml.Tag{
},
LanguageID: {
Name: "language",
Definition: "Specifies the language of the track in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageIETF Element is used in the same TrackEntry.",
Definition: "The language of the track, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageBCP47 Element is used in the same TrackEntry.",
Type: ebml.String,
},
LanguageIETFID: {
Name: "language_ietf",
Definition: "Specifies the language of the track according to [@!BCP47] and using the IANA Language Subtag Registry [@!IANALangRegistry]. If this Element is used, then any Language Elements used in the same TrackEntry **MUST** be ignored.",
LanguageBCP47ID: {
Name: "language_bcp47",
Definition: "The language of the track, in the [@!BCP47] form; see (#language-codes) on language codes. If this Element is used, then any Language Elements used in the same TrackEntry **MUST** be ignored.",
Type: ebml.String,
},
CodecIDID: {
Name: "codec_id",
Definition: "An ID corresponding to the codec, see [@!MatroskaCodec] for more info.",
Definition: "An ID corresponding to the codec, see [@?MatroskaCodec] for more info.",
Type: ebml.String,
},
CodecPrivateID: {
@ -879,7 +884,7 @@ var TrackEntry = ebml.Tag{
var BlockAdditionMapping = ebml.Tag{
BlockAddIDValueID: {
Name: "block_add_idvalue",
Definition: "If the track format extension needs content beside frames, the value refers to the BlockAddID ((#blockaddid-element)), value being described. To keep MaxBlockAdditionID as low as possible, small values **SHOULD** be used.",
Definition: "If the track format extension needs content beside frames, the value refers to the BlockAddID ((#blockaddid-element)), value being described.",
Type: ebml.Uinteger,
},
BlockAddIDNameID: {
@ -1045,7 +1050,7 @@ var Video = ebml.Tag{
},
OldStereoModeID: {
Name: "old_stereo_mode",
Definition: "DEPRECATED, DO NOT USE. Bogus StereoMode value used in old versions of libmatroska.",
Definition: "Bogus StereoMode value used in old versions of libmatroska.",
Type: ebml.Uinteger,
UintegerEnums: scalar.UToScalar{
0: {
@ -1311,7 +1316,7 @@ var Colour = ebml.Tag{
Sym: "unspecified",
},
3: {
Sym: "reserved",
Sym: "reserved2",
},
4: {
Sym: "gamma_2_2_curve_bt_470m",
@ -1375,7 +1380,7 @@ var Colour = ebml.Tag{
Sym: "unspecified",
},
3: {
Sym: "reserved",
Sym: "reserved2",
},
4: {
Sym: "itu_r_bt_470m",
@ -1429,42 +1434,42 @@ var Colour = ebml.Tag{
var MasteringMetadata = ebml.Tag{
PrimaryRChromaticityXID: {
Name: "primary_rchromaticity_x",
Definition: "Red X chromaticity coordinate, as defined by CIE 1931.",
Definition: "Red X chromaticity coordinate, as defined by [@!CIE-1931].",
Type: ebml.Float,
},
PrimaryRChromaticityYID: {
Name: "primary_rchromaticity_y",
Definition: "Red Y chromaticity coordinate, as defined by CIE 1931.",
Definition: "Red Y chromaticity coordinate, as defined by [@!CIE-1931].",
Type: ebml.Float,
},
PrimaryGChromaticityXID: {
Name: "primary_gchromaticity_x",
Definition: "Green X chromaticity coordinate, as defined by CIE 1931.",
Definition: "Green X chromaticity coordinate, as defined by [@!CIE-1931].",
Type: ebml.Float,
},
PrimaryGChromaticityYID: {
Name: "primary_gchromaticity_y",
Definition: "Green Y chromaticity coordinate, as defined by CIE 1931.",
Definition: "Green Y chromaticity coordinate, as defined by [@!CIE-1931].",
Type: ebml.Float,
},
PrimaryBChromaticityXID: {
Name: "primary_bchromaticity_x",
Definition: "Blue X chromaticity coordinate, as defined by CIE 1931.",
Definition: "Blue X chromaticity coordinate, as defined by [@!CIE-1931].",
Type: ebml.Float,
},
PrimaryBChromaticityYID: {
Name: "primary_bchromaticity_y",
Definition: "Blue Y chromaticity coordinate, as defined by CIE 1931.",
Definition: "Blue Y chromaticity coordinate, as defined by [@!CIE-1931].",
Type: ebml.Float,
},
WhitePointChromaticityXID: {
Name: "white_point_chromaticity_x",
Definition: "White X chromaticity coordinate, as defined by CIE 1931.",
Definition: "White X chromaticity coordinate, as defined by [@!CIE-1931].",
Type: ebml.Float,
},
WhitePointChromaticityYID: {
Name: "white_point_chromaticity_y",
Definition: "White Y chromaticity coordinate, as defined by CIE 1931.",
Definition: "White Y chromaticity coordinate, as defined by [@!CIE-1931].",
Type: ebml.Float,
},
LuminanceMaxID: {
@ -1547,6 +1552,63 @@ var Audio = ebml.Tag{
Definition: "Bits per sample, mostly used for PCM.",
Type: ebml.Uinteger,
},
EmphasisID: {
Name: "emphasis",
Definition: "Audio emphasis applied on audio samples. The player **MUST** apply the inverse emphasis to get the proper audio samples.",
Type: ebml.Uinteger,
UintegerEnums: scalar.UToScalar{
0: {
Sym: "no_emphasis",
},
1: {
Sym: "cd_audio",
Description: "First order filter with zero point at 50 microseconds and a pole at 15 microseconds. Also found on DVD Audio and MPEG audio.",
},
2: {
Sym: "reserved",
},
3: {
Sym: "ccit_j_17",
Description: "Defined in [@!ITU-J.17].",
},
4: {
Sym: "fm_50",
Description: "FM Radio in Europe. RC Filter with a time constant of 50 microseconds.",
},
5: {
Sym: "fm_75",
Description: "FM Radio in the USA. RC Filter with a time constant of 75 microseconds.",
},
10: {
Sym: "phono_riaa",
Description: "Phono filter with time constants of t1=3180, t2=318 and t3=75 microseconds. [@!NAB1964]",
},
11: {
Sym: "phono_iec_n78",
Description: "Phono filter with time constants of t1=3180, t2=450 and t3=50 microseconds.",
},
12: {
Sym: "phono_teldec",
Description: "Phono filter with time constants of t1=3180, t2=318 and t3=50 microseconds.",
},
13: {
Sym: "phono_emi",
Description: "Phono filter with time constants of t1=2500, t2=500 and t3=70 microseconds.",
},
14: {
Sym: "phono_columbia_lp",
Description: "Phono filter with time constants of t1=1590, t2=318 and t3=100 microseconds.",
},
15: {
Sym: "phono_london",
Description: "Phono filter with time constants of t1=1590, t2=318 and t3=50 microseconds.",
},
16: {
Sym: "phono_nartb",
Description: "Phono filter with time constants of t1=3180, t2=318 and t3=100 microseconds.",
},
},
},
}
var TrackOperation = ebml.Tag{
@ -1655,7 +1717,7 @@ var ContentEncoding = ebml.Tag{
},
ContentEncryptionID: {
Name: "content_encryption",
Definition: "Settings describing the encryption used. This Element **MUST** be present if the value of `ContentEncodingType` is 1 (encryption) and **MUST** be ignored otherwise.",
Definition: "Settings describing the encryption used. This Element **MUST** be present if the value of `ContentEncodingType` is 1 (encryption) and **MUST** be ignored otherwise. A Matroska Player **MAY** support encryption.",
Type: ebml.Master, Tag: ContentEncryption,
},
}
@ -1672,11 +1734,11 @@ var ContentCompression = ebml.Tag{
},
1: {
Sym: "bzlib",
Description: "bzip2 compression [@!BZIP2], **SHOULD NOT** be used; see usage notes.",
Description: "bzip2 compression [@?BZIP2], **SHOULD NOT** be used; see usage notes.",
},
2: {
Sym: "lzo1x",
Description: "Lempel-Ziv-Oberhumer compression [@!LZO], **SHOULD NOT** be used; see usage notes.",
Description: "Lempel-Ziv-Oberhumer compression [@?LZO], **SHOULD NOT** be used; see usage notes.",
},
3: {
Sym: "header_stripping",
@ -1694,31 +1756,32 @@ var ContentCompression = ebml.Tag{
var ContentEncryption = ebml.Tag{
ContentEncAlgoID: {
Name: "content_enc_algo",
Definition: "The encryption algorithm used. The value \"0\" means that the contents have not been encrypted.",
Definition: "The encryption algorithm used.",
Type: ebml.Uinteger,
UintegerEnums: scalar.UToScalar{
0: {
Sym: "not_encrypted",
Sym: "not_encrypted",
Description: "The data are not encrypted.",
},
1: {
Sym: "des",
Description: "Data Encryption Standard (DES) [@!FIPS.46-3].",
Description: "Data Encryption Standard (DES) [@?FIPS.46-3].",
},
2: {
Sym: "3des",
Description: "Triple Data Encryption Algorithm [@!SP.800-67].",
Description: "Triple Data Encryption Algorithm [@?SP.800-67].",
},
3: {
Sym: "twofish",
Description: "Twofish Encryption Algorithm [@!Twofish].",
Description: "Twofish Encryption Algorithm [@?Twofish].",
},
4: {
Sym: "blowfish",
Description: "Blowfish Encryption Algorithm [@!Blowfish].",
Description: "Blowfish Encryption Algorithm [@?Blowfish].",
},
5: {
Sym: "aes",
Description: "Advanced Encryption Standard (AES) [@!FIPS.197].",
Description: "Advanced Encryption Standard (AES) [@?FIPS.197].",
},
},
},
@ -1729,7 +1792,7 @@ var ContentEncryption = ebml.Tag{
},
ContentEncAESSettingsID: {
Name: "content_enc_aessettings",
Definition: "Settings describing the encryption algorithm used. It **MUST** be ignored if `ContentEncAlgo` is not AES (5).",
Definition: "Settings describing the encryption algorithm used.",
Type: ebml.Master, Tag: ContentEncAESSettings,
},
ContentSignatureID: {
@ -1776,16 +1839,16 @@ var ContentEncryption = ebml.Tag{
var ContentEncAESSettings = ebml.Tag{
AESSettingsCipherModeID: {
Name: "aessettings_cipher_mode",
Definition: "The AES cipher mode used in the encryption. It **MUST** be ignored if `ContentEncAlgo` is not AES (5).",
Definition: "The AES cipher mode used in the encryption.",
Type: ebml.Uinteger,
UintegerEnums: scalar.UToScalar{
1: {
Sym: "aes_ctr",
Description: "Counter [@!SP.800-38A].",
Description: "Counter [@?SP.800-38A].",
},
2: {
Sym: "aes_cbc",
Description: "Cipher Block Chaining [@!SP.800-38A].",
Description: "Cipher Block Chaining [@?SP.800-38A].",
},
},
},
@ -1820,7 +1883,7 @@ var CueTrackPositions = ebml.Tag{
},
CueClusterPositionID: {
Name: "cue_cluster_position",
Definition: "The Segment Position of the Cluster containing the associated Block.",
Definition: "The Segment Position ((#segment-position)) of the Cluster containing the associated Block.",
Type: ebml.Uinteger,
},
CueRelativePositionID: {
@ -1840,7 +1903,7 @@ var CueTrackPositions = ebml.Tag{
},
CueCodecStateID: {
Name: "cue_codec_state",
Definition: "The Segment Position of the Codec State corresponding to this Cue Element. 0 means that the data is taken from the initial Track Entry.",
Definition: "The Segment Position ((#segment-position)) of the Codec State corresponding to this Cue Element. 0 means that the data is taken from the initial Track Entry.",
Type: ebml.Uinteger,
},
CueReferenceID: {
@ -1892,9 +1955,9 @@ var AttachedFile = ebml.Tag{
Definition: "Filename of the attached file.",
Type: ebml.UTF8,
},
FileMimeTypeID: {
Name: "file_mime_type",
Definition: "MIME type of the file.",
FileMediaTypeID: {
Name: "file_media_type",
Definition: "Media type of the file following the [@!RFC6838] format.",
Type: ebml.String,
},
FileDataID: {
@ -1953,6 +2016,11 @@ var EditionEntry = ebml.Tag{
Definition: "Set to 1 if the chapters can be defined multiple times and the order to play them is enforced; see (#editionflagordered).",
Type: ebml.Uinteger,
},
EditionDisplayID: {
Name: "edition_display",
Definition: "Contains a possible string to use for the edition display for the given languages.",
Type: ebml.Master, Tag: EditionDisplay,
},
ChapterAtomID: {
Name: "chapter_atom",
Definition: "Contains the atom information to use as the chapter atom (apply to all tracks).",
@ -1960,6 +2028,19 @@ var EditionEntry = ebml.Tag{
},
}
var EditionDisplay = ebml.Tag{
EditionStringID: {
Name: "edition_string",
Definition: "Contains the string to use as the edition name.",
Type: ebml.UTF8,
},
EditionLanguageIETFID: {
Name: "edition_language_ietf",
Definition: "One language corresponding to the EditionString, in the [@!BCP47] form; see (#language-codes) on language codes.",
Type: ebml.String,
},
}
var ChapterAtom = ebml.Tag{
ChapterUIDID: {
Name: "chapter_uid",
@ -1968,7 +2049,7 @@ var ChapterAtom = ebml.Tag{
},
ChapterStringUIDID: {
Name: "chapter_string_uid",
Definition: "A unique string ID to identify the Chapter. Use for WebVTT cue identifier storage [@!WebVTT].",
Definition: "A unique string ID to identify the Chapter. Use for WebVTT cue identifier storage [@?WebVTT].",
Type: ebml.UTF8,
},
ChapterTimeStartID: {
@ -1991,14 +2072,49 @@ var ChapterAtom = ebml.Tag{
Definition: "Set to 1 if the chapter is enabled. It can be enabled/disabled by a Control Track. When disabled, the movie **SHOULD** skip all the content between the TimeStart and TimeEnd of this chapter; see (#chapter-flags) on Chapter flags.",
Type: ebml.Uinteger,
},
ChapterSegmentUIDID: {
Name: "chapter_segment_uid",
Definition: "The SegmentUID of another Segment to play during this chapter.",
ChapterSegmentUUIDID: {
Name: "chapter_segment_uuid",
Definition: "The SegmentUUID of another Segment to play during this chapter (128 bits). Like the SegmentUUID, it is a Universally Unique IDentifier stored in binary form [@!RFC4122].",
Type: ebml.Binary,
},
ChapterSkipTypeID: {
Name: "chapter_skip_type",
Definition: "Indicate what type of content the ChapterAtom contains and might be skipped. It can be used to automatically skip content based on the type. If a `ChapterAtom` is inside a `ChapterAtom` that has a `ChapterSkipType` set, it **MUST NOT** have a `ChapterSkipType` or have a `ChapterSkipType` with the same value as it's parent `ChapterAtom`. If the `ChapterAtom` doesn't contain a `ChapterTimeEnd`, the value of the `ChapterSkipType` is only valid until the next `ChapterAtom` with a `ChapterSkipType` value or the end of the file.",
Type: ebml.Uinteger,
UintegerEnums: scalar.UToScalar{
0: {
Sym: "no_skipping",
Description: "Content which should not be skipped.",
},
1: {
Sym: "opening_credits",
Description: "Credits usually found at the beginning of the content.",
},
2: {
Sym: "end_credits",
Description: "Credits usually found at the end of the content.",
},
3: {
Sym: "recap",
Description: "Recap of previous episodes of the content, usually found around the beginning.",
},
4: {
Sym: "next_preview",
Description: "Preview of the next episode of the content, usually found around the end. It may contain spoilers the user wants to avoid.",
},
5: {
Sym: "preview",
Description: "Preview of the current episode of the content, usually found around the beginning. It may contain spoilers the user want to avoid.",
},
6: {
Sym: "advertisement",
Description: "Advertisement within the content.",
},
},
},
ChapterSegmentEditionUIDID: {
Name: "chapter_segment_edition_uid",
Definition: "The EditionUID to play from the Segment linked in ChapterSegmentUID. If ChapterSegmentEditionUID is undeclared, then no Edition of the linked Segment is used; see (#medium-linking) on medium-linking Segments.",
Definition: "The EditionUID to play from the Segment linked in ChapterSegmentUUID. If ChapterSegmentEditionUID is undeclared, then no Edition of the linked Segment is used; see (#medium-linking) on medium-linking Segments.",
Type: ebml.Uinteger,
},
ChapterPhysicalEquivID: {
@ -2039,17 +2155,17 @@ var ChapterDisplay = ebml.Tag{
},
ChapLanguageID: {
Name: "chap_language",
Definition: "A language corresponding to the string, in the bibliographic ISO-639-2 form [@!ISO639-2]. This Element **MUST** be ignored if a ChapLanguageIETF Element is used within the same ChapterDisplay Element.",
Definition: "A language corresponding to the string, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if a ChapLanguageBCP47 Element is used within the same ChapterDisplay Element.",
Type: ebml.String,
},
ChapLanguageIETFID: {
Name: "chap_language_ietf",
Definition: "Specifies a language corresponding to the ChapString in the format defined in [@!BCP47] and using the IANA Language Subtag Registry [@!IANALangRegistry]. If a ChapLanguageIETF Element is used, then any ChapLanguage and ChapCountry Elements used in the same ChapterDisplay **MUST** be ignored.",
ChapLanguageBCP47ID: {
Name: "chap_language_bcp47",
Definition: "A language corresponding to the ChapString, in the [@!BCP47] form; see (#language-codes) on language codes. If a ChapLanguageBCP47 Element is used, then any ChapLanguage and ChapCountry Elements used in the same ChapterDisplay **MUST** be ignored.",
Type: ebml.String,
},
ChapCountryID: {
Name: "chap_country",
Definition: "A country corresponding to the string, using the same 2 octets country-codes as in Internet domains [@!IANADomains] based on [@!ISO3166-1] alpha-2 codes. This Element **MUST** be ignored if a ChapLanguageIETF Element is used within the same ChapterDisplay Element.",
Definition: "A country corresponding to the string, in the Matroska countries form; see (#country-codes) on country codes. This Element **MUST** be ignored if a ChapLanguageBCP47 Element is used within the same ChapterDisplay Element.",
Type: ebml.String,
},
}
@ -2155,71 +2271,71 @@ var Targets = ebml.Tag{
},
TargetTypeID: {
Name: "target_type",
Definition: "An informational string that can be used to display the logical level of the target like \"ALBUM\", \"TRACK\", \"MOVIE\", \"CHAPTER\", etc ; see Section 6.4 of [@!MatroskaTags].",
Definition: "An informational string that can be used to display the logical level of the target like \"ALBUM\", \"TRACK\", \"MOVIE\", \"CHAPTER\", etc ; see Section 6.4 of [@?MatroskaTags].",
Type: ebml.String,
StringEnums: scalar.StrToScalar{
"COLLECTION": {
Sym: "collection",
Sym: "targettypevalue_70",
},
"EDITION": {
Sym: "edition",
Sym: "targettypevalue_60",
},
"ISSUE": {
Sym: "issue",
Sym: "targettypevalue_60",
},
"VOLUME": {
Sym: "volume",
Sym: "targettypevalue_60",
},
"OPUS": {
Sym: "opus",
Sym: "targettypevalue_60",
},
"SEASON": {
Sym: "season",
Sym: "targettypevalue_60",
},
"SEQUEL": {
Sym: "sequel",
Sym: "targettypevalue_60",
},
"ALBUM": {
Sym: "album",
Sym: "targettypevalue_50",
},
"OPERA": {
Sym: "opera",
Sym: "targettypevalue_50",
},
"CONCERT": {
Sym: "concert",
Sym: "targettypevalue_50",
},
"MOVIE": {
Sym: "movie",
Sym: "targettypevalue_50",
},
"EPISODE": {
Sym: "episode",
Sym: "targettypevalue_50",
},
"PART": {
Sym: "part",
Sym: "targettypevalue_40",
},
"SESSION": {
Sym: "session",
Sym: "targettypevalue_40",
},
"TRACK": {
Sym: "track",
Sym: "targettypevalue_30",
},
"SONG": {
Sym: "song",
Sym: "targettypevalue_30",
},
"CHAPTER": {
Sym: "chapter",
Sym: "targettypevalue_30",
},
"SUBTRACK": {
Sym: "subtrack",
Sym: "targettypevalue_20",
},
"MOVEMENT": {
Sym: "movement",
Sym: "targettypevalue_20",
},
"SCENE": {
Sym: "scene",
Sym: "targettypevalue_20",
},
"SHOT": {
Sym: "shot",
Sym: "targettypevalue_10",
},
},
},
@ -2253,12 +2369,12 @@ var SimpleTag = ebml.Tag{
},
TagLanguageID: {
Name: "tag_language",
Definition: "Specifies the language of the tag specified, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the TagLanguageIETF Element is used within the same SimpleTag Element.",
Definition: "Specifies the language of the tag specified, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the TagLanguageBCP47 Element is used within the same SimpleTag Element.",
Type: ebml.String,
},
TagLanguageIETFID: {
Name: "tag_language_ietf",
Definition: "Specifies the language used in the TagString according to [@!BCP47] and using the IANA Language Subtag Registry [@!IANALangRegistry]. If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.",
TagLanguageBCP47ID: {
Name: "tag_language_bcp47",
Definition: "The language used in the TagString, in the [@!BCP47] form; see (#language-codes) on language codes. If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.",
Type: ebml.String,
},
TagDefaultID: {

2
format/matroska/matroska.go
View File

@ -27,6 +27,7 @@ import (
)
//go:embed matroska.jq
//go:embed matroska.md
var matroskaFS embed.FS
var aacFrameFormat decode.Group
@ -78,7 +79,6 @@ func init() {
{Names: []string{format.VP9_CFM}, Group: &vp9CFMFormat},
{Names: []string{format.VP9_FRAME}, Group: &vp9FrameFormat},
},
Functions: []string{"_help"},
})
interp.RegisterFS(matroskaFS)

14
format/matroska/matroska.jq
View File

@ -12,17 +12,3 @@ def matroska_path:
| format_root
| matroska_path($c)
);
def _matroska__help:
{ examples: [
{comment: "Lookup element decode value using `matroska_path`", expr: "matroska_path(\".Segment.Tracks[0)\""},
{comment: "Return `matroska_path` string for a box decode value", expr: "grep_by(.id == \"Tracks\") | matroska_path"}
],
links: [
{url: "https://tools.ietf.org/html/draft-ietf-cellar-ebml-00"},
{url: "https://matroska.org/technical/specs/index.html"},
{url: "https://www.matroska.org/technical/basics.html"},
{url: "https://www.matroska.org/technical/codec_specs.html"},
{url: "https://wiki.xiph.org/MatroskaOpus"}
]
};

18
format/matroska/matroska.md Normal file
View File

@ -0,0 +1,18 @@
### Lookup element using path
```sh
$ fq 'matroska_path(".Segment.Tracks[0)")' file.mkv
```
### Get path to element
```sh
$ fq 'grep_by(.id == "Tracks") | matroska_path' file.mkv
```
### References
- https://tools.ietf.org/html/draft-ietf-cellar-ebml-00
- https://matroska.org/technical/specs/index.html
- https://www.matroska.org/technical/basics.html
- https://www.matroska.org/technical/codec_specs.html
- https://wiki.xiph.org/MatroskaOpus

28
format/matroska/testdata/aac.fqtest vendored
View File

@ -49,7 +49,7 @@ $ fq -d matroska dv aac.mkv
0x030|00 00 04 90 |.... |
| | | elements[0:7]: 0x34-0x4c3.7 (1168)
| | | [0]{}: element 0x34-0x78.7 (69)
0x030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains the Segment Position of other Top-Level Elements.) 0x34-0x37.7 (4)
0x030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains seeking information of Top-Level Elements; see (#data-layout).) 0x34-0x37.7 (4)
| | | type: "master" 0x38-NA (0)
0x030| c0 | . | size: 64 0x38-0x38.7 (1)
| | | elements[0:5]: 0x39-0x78.7 (64)
@ -65,12 +65,12 @@ $ fq -d matroska dv aac.mkv
0x040| 8b | . | size: 11 0x41-0x41.7 (1)
| | | elements[0:2]: 0x42-0x4c.7 (11)
| | | [0]{}: element 0x42-0x48.7 (7)
0x040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x42-0x43.7 (2)
0x040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x42-0x43.7 (2)
| | | type: "binary" 0x44-NA (0)
0x040| 84 | . | size: 4 0x44-0x44.7 (1)
0x040| 15 49 a9 66 | .I.f | value: raw bits 0x45-0x48.7 (4)
| | | [1]{}: element 0x49-0x4c.7 (4)
0x040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x49-0x4a.7 (2)
0x040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x49-0x4a.7 (2)
| | | type: "uinteger" 0x4b-NA (0)
0x040| 81 | . | size: 1 0x4b-0x4b.7 (1)
0x040| a1 | . | value: 161 0x4c-0x4c.7 (1)
@ -80,12 +80,12 @@ $ fq -d matroska dv aac.mkv
0x040| 8b| .| size: 11 0x4f-0x4f.7 (1)
| | | elements[0:2]: 0x50-0x5a.7 (11)
| | | [0]{}: element 0x50-0x56.7 (7)
0x050|53 ab |S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x50-0x51.7 (2)
0x050|53 ab |S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x50-0x51.7 (2)
| | | type: "binary" 0x52-NA (0)
0x050| 84 | . | size: 4 0x52-0x52.7 (1)
0x050| 16 54 ae 6b | .T.k | value: raw bits 0x53-0x56.7 (4)
| | | [1]{}: element 0x57-0x5a.7 (4)
0x050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x57-0x58.7 (2)
0x050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x57-0x58.7 (2)
| | | type: "uinteger" 0x59-NA (0)
0x050| 81 | . | size: 1 0x59-0x59.7 (1)
0x050| f1 | . | value: 241 0x5a-0x5a.7 (1)
@ -95,12 +95,12 @@ $ fq -d matroska dv aac.mkv
0x050| 8c | . | size: 12 0x5d-0x5d.7 (1)
| | | elements[0:2]: 0x5e-0x69.7 (12)
| | | [0]{}: element 0x5e-0x64.7 (7)
0x050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x5e-0x5f.7 (2)
0x050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x5e-0x5f.7 (2)
| | | type: "binary" 0x60-NA (0)
0x060|84 |. | size: 4 0x60-0x60.7 (1)
0x060| 12 54 c3 67 | .T.g | value: raw bits 0x61-0x64.7 (4)
| | | [1]{}: element 0x65-0x69.7 (5)
0x060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x65-0x66.7 (2)
0x060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x65-0x66.7 (2)
| | | type: "uinteger" 0x67-NA (0)
0x060| 82 | . | size: 2 0x67-0x67.7 (1)
0x060| 01 42 | .B | value: 322 0x68-0x69.7 (2)
@ -110,12 +110,12 @@ $ fq -d matroska dv aac.mkv
0x060| 8c | . | size: 12 0x6c-0x6c.7 (1)
| | | elements[0:2]: 0x6d-0x78.7 (12)
| | | [0]{}: element 0x6d-0x73.7 (7)
0x060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x6d-0x6e.7 (2)
0x060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x6d-0x6e.7 (2)
| | | type: "binary" 0x6f-NA (0)
0x060| 84| .| size: 4 0x6f-0x6f.7 (1)
0x070|1c 53 bb 6b |.S.k | value: raw bits 0x70-0x73.7 (4)
| | | [1]{}: element 0x74-0x78.7 (5)
0x070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x74-0x75.7 (2)
0x070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x74-0x75.7 (2)
| | | type: "uinteger" 0x76-NA (0)
0x070| 82 | . | size: 2 0x76-0x76.7 (1)
0x070| 04 74 | .t | value: 1140 0x77-0x78.7 (2)
@ -155,7 +155,7 @@ $ fq -d matroska dv aac.mkv
0x0f0| 4c 61 76 66 35 38| Lavf58| value: "Lavf58.45.100" 0xfa-0x106.7 (13)
0x100|2e 34 35 2e 31 30 30 |.45.100 |
| | | [4]{}: element 0x107-0x119.7 (19)
0x100| 73 a4 | s. | id: "segment_uid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits).) 0x107-0x108.7 (2)
0x100| 73 a4 | s. | id: "segment_uuid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits). It is effectively a Universally Unique IDentifier stored in binary form [@!RFC4122].) 0x107-0x108.7 (2)
| | | type: "binary" 0x109-NA (0)
0x100| 90 | . | size: 16 0x109-0x109.7 (1)
0x100| 17 bd f9 34 6e 97| ...4n.| value: raw bits 0x10a-0x119.7 (16)
@ -198,13 +198,13 @@ $ fq -d matroska dv aac.mkv
0x140| 81 | . | size: 1 0x148-0x148.7 (1)
0x140| 00 | . | value: 0 0x149-0x149.7 (1)
| | | [3]{}: element 0x14a-0x150.7 (7)
0x140| 22 b5 9c | ".. | id: "language" (0x22b59c) (Specifies the language of the track in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageIETF Element is used in the same TrackEntry.) 0x14a-0x14c.7 (3)
0x140| 22 b5 9c | ".. | id: "language" (0x22b59c) (The language of the track, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageBCP47 Element is used in the same TrackEntry.) 0x14a-0x14c.7 (3)
| | | type: "string" 0x14d-NA (0)
0x140| 83 | . | size: 3 0x14d-0x14d.7 (1)
0x140| 75 6e| un| value: "und" 0x14e-0x150.7 (3)
0x150|64 |d |
| | | [4]{}: element 0x151-0x157.7 (7)
0x150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@!MatroskaCodec] for more info.) 0x151-0x151.7 (1)
0x150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@?MatroskaCodec] for more info.) 0x151-0x151.7 (1)
| | | type: "string" 0x152-NA (0)
0x150| 85 | . | size: 5 0x152-0x152.7 (1)
0x150| 41 5f 41 41 43 | A_AAC | value: "A_AAC" 0x153-0x157.7 (5)
@ -243,7 +243,7 @@ $ fq -d matroska dv aac.mkv
0x170| 08 | . | channel_configuration: 1 (front-center) 0x172.1-0x172.4 (0.4)
0x170| 08 56 e5 00 | .V.. | var_aot_or_byte_align: raw bits 0x172.5-0x175.7 (3.3)
| | | [4]{}: element 0x176-0x217.7 (162)
0x170| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@!MatroskaTags].) 0x176-0x179.7 (4)
0x170| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@?MatroskaTags].) 0x176-0x179.7 (4)
| | | type: "master" 0x17a-NA (0)
0x170| 40 9c | @. | size: 156 0x17a-0x17b.7 (2)
| | | elements[0:3]: 0x17c-0x217.7 (156)
@ -497,7 +497,7 @@ $ fq -d matroska dv aac.mkv
0x4b0| 81 | . | size: 1 0x4bb-0x4bb.7 (1)
0x4b0| 01 | . | value: 1 0x4bc-0x4bc.7 (1)
| | | [1]{}: element 0x4bd-0x4c0.7 (4)
0x4b0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position of the Cluster containing the associated Block.) 0x4bd-0x4bd.7 (1)
0x4b0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position ((#segment-position)) of the Cluster containing the associated Block.) 0x4bd-0x4bd.7 (1)
| | | type: "uinteger" 0x4be-NA (0)
0x4b0| 82 | . | size: 2 0x4be-0x4be.7 (1)
0x4b0| 01| .| value: 484 0x4bf-0x4c0.7 (2)

28
format/matroska/testdata/av1.fqtest vendored
View File

@ -49,7 +49,7 @@ $ fq -d matroska dv av1.mkv
0x0030|00 00 13 b3 |.... |
| | | elements[0:7]: 0x34-0x13e6.7 (5043)
| | | [0]{}: element 0x34-0x78.7 (69)
0x0030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains the Segment Position of other Top-Level Elements.) 0x34-0x37.7 (4)
0x0030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains seeking information of Top-Level Elements; see (#data-layout).) 0x34-0x37.7 (4)
| | | type: "master" 0x38-NA (0)
0x0030| c0 | . | size: 64 0x38-0x38.7 (1)
| | | elements[0:5]: 0x39-0x78.7 (64)
@ -65,12 +65,12 @@ $ fq -d matroska dv av1.mkv
0x0040| 8b | . | size: 11 0x41-0x41.7 (1)
| | | elements[0:2]: 0x42-0x4c.7 (11)
| | | [0]{}: element 0x42-0x48.7 (7)
0x0040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x42-0x43.7 (2)
0x0040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x42-0x43.7 (2)
| | | type: "binary" 0x44-NA (0)
0x0040| 84 | . | size: 4 0x44-0x44.7 (1)
0x0040| 15 49 a9 66 | .I.f | value: raw bits 0x45-0x48.7 (4)
| | | [1]{}: element 0x49-0x4c.7 (4)
0x0040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x49-0x4a.7 (2)
0x0040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x49-0x4a.7 (2)
| | | type: "uinteger" 0x4b-NA (0)
0x0040| 81 | . | size: 1 0x4b-0x4b.7 (1)
0x0040| a1 | . | value: 161 0x4c-0x4c.7 (1)
@ -80,12 +80,12 @@ $ fq -d matroska dv av1.mkv
0x0040| 8b| .| size: 11 0x4f-0x4f.7 (1)
| | | elements[0:2]: 0x50-0x5a.7 (11)
| | | [0]{}: element 0x50-0x56.7 (7)
0x0050|53 ab |S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x50-0x51.7 (2)
0x0050|53 ab |S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x50-0x51.7 (2)
| | | type: "binary" 0x52-NA (0)
0x0050| 84 | . | size: 4 0x52-0x52.7 (1)
0x0050| 16 54 ae 6b | .T.k | value: raw bits 0x53-0x56.7 (4)
| | | [1]{}: element 0x57-0x5a.7 (4)
0x0050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x57-0x58.7 (2)
0x0050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x57-0x58.7 (2)
| | | type: "uinteger" 0x59-NA (0)
0x0050| 81 | . | size: 1 0x59-0x59.7 (1)
0x0050| f1 | . | value: 241 0x5a-0x5a.7 (1)
@ -95,12 +95,12 @@ $ fq -d matroska dv av1.mkv
0x0050| 8c | . | size: 12 0x5d-0x5d.7 (1)
| | | elements[0:2]: 0x5e-0x69.7 (12)
| | | [0]{}: element 0x5e-0x64.7 (7)
0x0050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x5e-0x5f.7 (2)
0x0050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x5e-0x5f.7 (2)
| | | type: "binary" 0x60-NA (0)
0x0060|84 |. | size: 4 0x60-0x60.7 (1)
0x0060| 12 54 c3 67 | .T.g | value: raw bits 0x61-0x64.7 (4)
| | | [1]{}: element 0x65-0x69.7 (5)
0x0060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x65-0x66.7 (2)
0x0060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x65-0x66.7 (2)
| | | type: "uinteger" 0x67-NA (0)
0x0060| 82 | . | size: 2 0x67-0x67.7 (1)
0x0060| 01 46 | .F | value: 326 0x68-0x69.7 (2)
@ -110,12 +110,12 @@ $ fq -d matroska dv av1.mkv
0x0060| 8c | . | size: 12 0x6c-0x6c.7 (1)
| | | elements[0:2]: 0x6d-0x78.7 (12)
| | | [0]{}: element 0x6d-0x73.7 (7)
0x0060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x6d-0x6e.7 (2)
0x0060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x6d-0x6e.7 (2)
| | | type: "binary" 0x6f-NA (0)
0x0060| 84| .| size: 4 0x6f-0x6f.7 (1)
0x0070|1c 53 bb 6b |.S.k | value: raw bits 0x70-0x73.7 (4)
| | | [1]{}: element 0x74-0x78.7 (5)
0x0070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x74-0x75.7 (2)
0x0070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x74-0x75.7 (2)
| | | type: "uinteger" 0x76-NA (0)
0x0070| 82 | . | size: 2 0x76-0x76.7 (1)
0x0070| 13 97 | .. | value: 5015 0x77-0x78.7 (2)
@ -155,7 +155,7 @@ $ fq -d matroska dv av1.mkv
0x00f0| 4c 61 76 66 35 38| Lavf58| value: "Lavf58.45.100" 0xfa-0x106.7 (13)
0x0100|2e 34 35 2e 31 30 30 |.45.100 |
| | | [4]{}: element 0x107-0x119.7 (19)
0x0100| 73 a4 | s. | id: "segment_uid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits).) 0x107-0x108.7 (2)
0x0100| 73 a4 | s. | id: "segment_uuid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits). It is effectively a Universally Unique IDentifier stored in binary form [@!RFC4122].) 0x107-0x108.7 (2)
| | | type: "binary" 0x109-NA (0)
0x0100| 90 | . | size: 16 0x109-0x109.7 (1)
0x0100| 46 bd 6f 31 1c 40| F.o1.@| value: raw bits 0x10a-0x119.7 (16)
@ -198,13 +198,13 @@ $ fq -d matroska dv av1.mkv
0x0140| 81 | . | size: 1 0x148-0x148.7 (1)
0x0140| 00 | . | value: 0 0x149-0x149.7 (1)
| | | [3]{}: element 0x14a-0x150.7 (7)
0x0140| 22 b5 9c | ".. | id: "language" (0x22b59c) (Specifies the language of the track in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageIETF Element is used in the same TrackEntry.) 0x14a-0x14c.7 (3)
0x0140| 22 b5 9c | ".. | id: "language" (0x22b59c) (The language of the track, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageBCP47 Element is used in the same TrackEntry.) 0x14a-0x14c.7 (3)
| | | type: "string" 0x14d-NA (0)
0x0140| 83 | . | size: 3 0x14d-0x14d.7 (1)
0x0140| 75 6e| un| value: "und" 0x14e-0x150.7 (3)
0x0150|64 |d |
| | | [4]{}: element 0x151-0x157.7 (7)
0x0150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@!MatroskaCodec] for more info.) 0x151-0x151.7 (1)
0x0150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@?MatroskaCodec] for more info.) 0x151-0x151.7 (1)
| | | type: "string" 0x152-NA (0)
0x0150| 85 | . | size: 5 0x152-0x152.7 (1)
0x0150| 56 5f 41 56 31 | V_AV1 | value: "V_AV1" 0x153-0x157.7 (5)
@ -254,7 +254,7 @@ $ fq -d matroska dv av1.mkv
0x0170| 00 | . | initial_presentation_delay_present: false 0x179.3-0x179.3 (0.1)
0x0170| 00 | . | reserved: 0 0x179.4-0x179.7 (0.4)
| | | [4]{}: element 0x17a-0x220.7 (167)
0x0170| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@!MatroskaTags].) 0x17a-0x17d.7 (4)
0x0170| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@?MatroskaTags].) 0x17a-0x17d.7 (4)
| | | type: "master" 0x17e-NA (0)
0x0170| 40 a1| @.| size: 161 0x17e-0x17f.7 (2)
| | | elements[0:3]: 0x180-0x220.7 (161)
@ -431,7 +431,7 @@ $ fq -d matroska dv av1.mkv
0x13d0| 81 | . | size: 1 0x13de-0x13de.7 (1)
0x13d0| 01| .| value: 1 0x13df-0x13df.7 (1)
| | | [1]{}: element 0x13e0-0x13e3.7 (4)
0x13e0|f1 |. | id: "cue_cluster_position" (0xf1) (The Segment Position of the Cluster containing the associated Block.) 0x13e0-0x13e0.7 (1)
0x13e0|f1 |. | id: "cue_cluster_position" (0xf1) (The Segment Position ((#segment-position)) of the Cluster containing the associated Block.) 0x13e0-0x13e0.7 (1)
| | | type: "uinteger" 0x13e1-NA (0)
0x13e0| 82 | . | size: 2 0x13e1-0x13e1.7 (1)
0x13e0| 01 ed | .. | value: 493 0x13e2-0x13e3.7 (2)

28
format/matroska/testdata/avc.fqtest vendored
View File

@ -49,7 +49,7 @@ $ fq -d matroska dv avc.mkv
0x00030|00 00 0d 13 |.... |
| | | elements[0:7]: 0x34-0xd46.7 (3347)
| | | [0]{}: element 0x34-0x78.7 (69)
0x00030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains the Segment Position of other Top-Level Elements.) 0x34-0x37.7 (4)
0x00030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains seeking information of Top-Level Elements; see (#data-layout).) 0x34-0x37.7 (4)
| | | type: "master" 0x38-NA (0)
0x00030| c0 | . | size: 64 0x38-0x38.7 (1)
| | | elements[0:5]: 0x39-0x78.7 (64)
@ -65,12 +65,12 @@ $ fq -d matroska dv avc.mkv
0x00040| 8b | . | size: 11 0x41-0x41.7 (1)
| | | elements[0:2]: 0x42-0x4c.7 (11)
| | | [0]{}: element 0x42-0x48.7 (7)
0x00040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x42-0x43.7 (2)
0x00040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x42-0x43.7 (2)
| | | type: "binary" 0x44-NA (0)
0x00040| 84 | . | size: 4 0x44-0x44.7 (1)
0x00040| 15 49 a9 66 | .I.f | value: raw bits 0x45-0x48.7 (4)
| | | [1]{}: element 0x49-0x4c.7 (4)
0x00040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x49-0x4a.7 (2)
0x00040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x49-0x4a.7 (2)
| | | type: "uinteger" 0x4b-NA (0)
0x00040| 81 | . | size: 1 0x4b-0x4b.7 (1)
0x00040| a1 | . | value: 161 0x4c-0x4c.7 (1)
@ -80,12 +80,12 @@ $ fq -d matroska dv avc.mkv
0x00040| 8b| .| size: 11 0x4f-0x4f.7 (1)
| | | elements[0:2]: 0x50-0x5a.7 (11)
| | | [0]{}: element 0x50-0x56.7 (7)
0x00050|53 ab |S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x50-0x51.7 (2)
0x00050|53 ab |S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x50-0x51.7 (2)
| | | type: "binary" 0x52-NA (0)
0x00050| 84 | . | size: 4 0x52-0x52.7 (1)
0x00050| 16 54 ae 6b | .T.k | value: raw bits 0x53-0x56.7 (4)
| | | [1]{}: element 0x57-0x5a.7 (4)
0x00050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x57-0x58.7 (2)
0x00050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x57-0x58.7 (2)
| | | type: "uinteger" 0x59-NA (0)
0x00050| 81 | . | size: 1 0x59-0x59.7 (1)
0x00050| f1 | . | value: 241 0x5a-0x5a.7 (1)
@ -95,12 +95,12 @@ $ fq -d matroska dv avc.mkv
0x00050| 8c | . | size: 12 0x5d-0x5d.7 (1)
| | | elements[0:2]: 0x5e-0x69.7 (12)
| | | [0]{}: element 0x5e-0x64.7 (7)
0x00050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x5e-0x5f.7 (2)
0x00050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x5e-0x5f.7 (2)
| | | type: "binary" 0x60-NA (0)
0x00060|84 |. | size: 4 0x60-0x60.7 (1)
0x00060| 12 54 c3 67 | .T.g | value: raw bits 0x61-0x64.7 (4)
| | | [1]{}: element 0x65-0x69.7 (5)
0x00060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x65-0x66.7 (2)
0x00060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x65-0x66.7 (2)
| | | type: "uinteger" 0x67-NA (0)
0x00060| 82 | . | size: 2 0x67-0x67.7 (1)
0x00060| 01 7b | .{ | value: 379 0x68-0x69.7 (2)
@ -110,12 +110,12 @@ $ fq -d matroska dv avc.mkv
0x00060| 8c | . | size: 12 0x6c-0x6c.7 (1)
| | | elements[0:2]: 0x6d-0x78.7 (12)
| | | [0]{}: element 0x6d-0x73.7 (7)
0x00060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x6d-0x6e.7 (2)
0x00060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x6d-0x6e.7 (2)
| | | type: "binary" 0x6f-NA (0)
0x00060| 84| .| size: 4 0x6f-0x6f.7 (1)
0x00070|1c 53 bb 6b |.S.k | value: raw bits 0x70-0x73.7 (4)
| | | [1]{}: element 0x74-0x78.7 (5)
0x00070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x74-0x75.7 (2)
0x00070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x74-0x75.7 (2)
| | | type: "uinteger" 0x76-NA (0)
0x00070| 82 | . | size: 2 0x76-0x76.7 (1)
0x00070| 0c f7 | .. | value: 3319 0x77-0x78.7 (2)
@ -155,7 +155,7 @@ $ fq -d matroska dv avc.mkv
0x000f0| 4c 61 76 66 35 38| Lavf58| value: "Lavf58.45.100" 0xfa-0x106.7 (13)
0x00100|2e 34 35 2e 31 30 30 |.45.100 |
| | | [4]{}: element 0x107-0x119.7 (19)
0x00100| 73 a4 | s. | id: "segment_uid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits).) 0x107-0x108.7 (2)
0x00100| 73 a4 | s. | id: "segment_uuid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits). It is effectively a Universally Unique IDentifier stored in binary form [@!RFC4122].) 0x107-0x108.7 (2)
| | | type: "binary" 0x109-NA (0)
0x00100| 90 | . | size: 16 0x109-0x109.7 (1)
0x00100| 7c 01 b0 2d a3 3f| |..-.?| value: raw bits 0x10a-0x119.7 (16)
@ -198,13 +198,13 @@ $ fq -d matroska dv avc.mkv
0x00140| 81 | . | size: 1 0x149-0x149.7 (1)
0x00140| 00 | . | value: 0 0x14a-0x14a.7 (1)
| | | [3]{}: element 0x14b-0x151.7 (7)
0x00140| 22 b5 9c | ".. | id: "language" (0x22b59c) (Specifies the language of the track in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageIETF Element is used in the same TrackEntry.) 0x14b-0x14d.7 (3)
0x00140| 22 b5 9c | ".. | id: "language" (0x22b59c) (The language of the track, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageBCP47 Element is used in the same TrackEntry.) 0x14b-0x14d.7 (3)
| | | type: "string" 0x14e-NA (0)
0x00140| 83 | . | size: 3 0x14e-0x14e.7 (1)
0x00140| 75| u| value: "und" 0x14f-0x151.7 (3)
0x00150|6e 64 |nd |
| | | [4]{}: element 0x152-0x162.7 (17)
0x00150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@!MatroskaCodec] for more info.) 0x152-0x152.7 (1)
0x00150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@?MatroskaCodec] for more info.) 0x152-0x152.7 (1)
| | | type: "string" 0x153-NA (0)
0x00150| 8f | . | size: 15 0x153-0x153.7 (1)
0x00150| 56 5f 4d 50 45 47 34 2f 49 53 4f 2f| V_MPEG4/ISO/| value: "V_MPEG4/ISO/AVC" 0x154-0x162.7 (15)
@ -340,7 +340,7 @@ $ fq -d matroska dv avc.mkv
0x001a0| eb e3 c4 48 44 | ...HD | data: raw bits 0x1a6-0x1aa.7 (5)
0x001a0| ff f8 f8 00 | .... | data: raw bits 0x1ab-0x1ae.7 (4)
| | | [4]{}: element 0x1af-0x254.7 (166)
0x001a0| 12| .| id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@!MatroskaTags].) 0x1af-0x1b2.7 (4)
0x001a0| 12| .| id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@?MatroskaTags].) 0x1af-0x1b2.7 (4)
0x001b0|54 c3 67 |T.g |
| | | type: "master" 0x1b3-NA (0)
0x001b0| 40 a0 | @. | size: 160 0x1b3-0x1b4.7 (2)
@ -517,7 +517,7 @@ $ fq -d matroska dv avc.mkv
0x00d30| 81 | . | size: 1 0xd3e-0xd3e.7 (1)
0x00d30| 01| .| value: 1 0xd3f-0xd3f.7 (1)
| | | [1]{}: element 0xd40-0xd43.7 (4)
0x00d40|f1 |. | id: "cue_cluster_position" (0xf1) (The Segment Position of the Cluster containing the associated Block.) 0xd40-0xd40.7 (1)
0x00d40|f1 |. | id: "cue_cluster_position" (0xf1) (The Segment Position ((#segment-position)) of the Cluster containing the associated Block.) 0xd40-0xd40.7 (1)
| | | type: "uinteger" 0xd41-NA (0)
0x00d40| 82 | . | size: 2 0xd41-0xd41.7 (1)
0x00d40| 02 21 | .! | value: 545 0xd42-0xd43.7 (2)

28
format/matroska/testdata/flac.fqtest vendored
View File

@ -49,7 +49,7 @@ $ fq -d matroska dv flac.mkv
0x030|00 00 04 9b |.... |
| | | elements[0:7]: 0x34-0x4ce.7 (1179)
| | | [0]{}: element 0x34-0x78.7 (69)
0x030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains the Segment Position of other Top-Level Elements.) 0x34-0x37.7 (4)
0x030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains seeking information of Top-Level Elements; see (#data-layout).) 0x34-0x37.7 (4)
| | | type: "master" 0x38-NA (0)
0x030| c0 | . | size: 64 0x38-0x38.7 (1)
| | | elements[0:5]: 0x39-0x78.7 (64)
@ -65,12 +65,12 @@ $ fq -d matroska dv flac.mkv
0x040| 8b | . | size: 11 0x41-0x41.7 (1)
| | | elements[0:2]: 0x42-0x4c.7 (11)
| | | [0]{}: element 0x42-0x48.7 (7)
0x040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x42-0x43.7 (2)
0x040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x42-0x43.7 (2)
| | | type: "binary" 0x44-NA (0)
0x040| 84 | . | size: 4 0x44-0x44.7 (1)
0x040| 15 49 a9 66 | .I.f | value: raw bits 0x45-0x48.7 (4)
| | | [1]{}: element 0x49-0x4c.7 (4)
0x040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x49-0x4a.7 (2)
0x040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x49-0x4a.7 (2)
| | | type: "uinteger" 0x4b-NA (0)
0x040| 81 | . | size: 1 0x4b-0x4b.7 (1)
0x040| a1 | . | value: 161 0x4c-0x4c.7 (1)
@ -80,12 +80,12 @@ $ fq -d matroska dv flac.mkv
0x040| 8b| .| size: 11 0x4f-0x4f.7 (1)
| | | elements[0:2]: 0x50-0x5a.7 (11)
| | | [0]{}: element 0x50-0x56.7 (7)
0x050|53 ab |S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x50-0x51.7 (2)
0x050|53 ab |S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x50-0x51.7 (2)
| | | type: "binary" 0x52-NA (0)
0x050| 84 | . | size: 4 0x52-0x52.7 (1)
0x050| 16 54 ae 6b | .T.k | value: raw bits 0x53-0x56.7 (4)
| | | [1]{}: element 0x57-0x5a.7 (4)
0x050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x57-0x58.7 (2)
0x050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x57-0x58.7 (2)
| | | type: "uinteger" 0x59-NA (0)
0x050| 81 | . | size: 1 0x59-0x59.7 (1)
0x050| f1 | . | value: 241 0x5a-0x5a.7 (1)
@ -95,12 +95,12 @@ $ fq -d matroska dv flac.mkv
0x050| 8c | . | size: 12 0x5d-0x5d.7 (1)
| | | elements[0:2]: 0x5e-0x69.7 (12)
| | | [0]{}: element 0x5e-0x64.7 (7)
0x050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x5e-0x5f.7 (2)
0x050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x5e-0x5f.7 (2)
| | | type: "binary" 0x60-NA (0)
0x060|84 |. | size: 4 0x60-0x60.7 (1)
0x060| 12 54 c3 67 | .T.g | value: raw bits 0x61-0x64.7 (4)
| | | [1]{}: element 0x65-0x69.7 (5)
0x060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x65-0x66.7 (2)
0x060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x65-0x66.7 (2)
| | | type: "uinteger" 0x67-NA (0)
0x060| 82 | . | size: 2 0x67-0x67.7 (1)
0x060| 01 68 | .h | value: 360 0x68-0x69.7 (2)
@ -110,12 +110,12 @@ $ fq -d matroska dv flac.mkv
0x060| 8c | . | size: 12 0x6c-0x6c.7 (1)
| | | elements[0:2]: 0x6d-0x78.7 (12)
| | | [0]{}: element 0x6d-0x73.7 (7)
0x060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x6d-0x6e.7 (2)
0x060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x6d-0x6e.7 (2)
| | | type: "binary" 0x6f-NA (0)
0x060| 84| .| size: 4 0x6f-0x6f.7 (1)
0x070|1c 53 bb 6b |.S.k | value: raw bits 0x70-0x73.7 (4)
| | | [1]{}: element 0x74-0x78.7 (5)
0x070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x74-0x75.7 (2)
0x070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x74-0x75.7 (2)
| | | type: "uinteger" 0x76-NA (0)
0x070| 82 | . | size: 2 0x76-0x76.7 (1)
0x070| 04 7f | .. | value: 1151 0x77-0x78.7 (2)
@ -155,7 +155,7 @@ $ fq -d matroska dv flac.mkv
0x0f0| 4c 61 76 66 35 38| Lavf58| value: "Lavf58.45.100" 0xfa-0x106.7 (13)
0x100|2e 34 35 2e 31 30 30 |.45.100 |
| | | [4]{}: element 0x107-0x119.7 (19)
0x100| 73 a4 | s. | id: "segment_uid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits).) 0x107-0x108.7 (2)
0x100| 73 a4 | s. | id: "segment_uuid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits). It is effectively a Universally Unique IDentifier stored in binary form [@!RFC4122].) 0x107-0x108.7 (2)
| | | type: "binary" 0x109-NA (0)
0x100| 90 | . | size: 16 0x109-0x109.7 (1)
0x100| 57 0f ef b3 6f 8c| W...o.| value: raw bits 0x10a-0x119.7 (16)
@ -198,13 +198,13 @@ $ fq -d matroska dv flac.mkv
0x140| 81 | . | size: 1 0x148-0x148.7 (1)
0x140| 00 | . | value: 0 0x149-0x149.7 (1)
| | | [3]{}: element 0x14a-0x150.7 (7)
0x140| 22 b5 9c | ".. | id: "language" (0x22b59c) (Specifies the language of the track in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageIETF Element is used in the same TrackEntry.) 0x14a-0x14c.7 (3)
0x140| 22 b5 9c | ".. | id: "language" (0x22b59c) (The language of the track, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageBCP47 Element is used in the same TrackEntry.) 0x14a-0x14c.7 (3)
| | | type: "string" 0x14d-NA (0)
0x140| 83 | . | size: 3 0x14d-0x14d.7 (1)
0x140| 75 6e| un| value: "und" 0x14e-0x150.7 (3)
0x150|64 |d |
| | | [4]{}: element 0x151-0x158.7 (8)
0x150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@!MatroskaCodec] for more info.) 0x151-0x151.7 (1)
0x150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@?MatroskaCodec] for more info.) 0x151-0x151.7 (1)
| | | type: "string" 0x152-NA (0)
0x150| 86 | . | size: 6 0x152-0x152.7 (1)
0x150| 41 5f 46 4c 41 43 | A_FLAC | value: "A_FLAC" 0x153-0x158.7 (6)
@ -257,7 +257,7 @@ $ fq -d matroska dv flac.mkv
0x180| e9 16 ab 02| ....| md5: "e916ab02137281386a28174fe11bffec" (raw bits) 0x18c-0x19b.7 (16)
0x190|13 72 81 38 6a 28 17 4f e1 1b ff ec |.r.8j(.O.... |
| | | [4]{}: element 0x19c-0x23e.7 (163)
0x190| 12 54 c3 67| .T.g| id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@!MatroskaTags].) 0x19c-0x19f.7 (4)
0x190| 12 54 c3 67| .T.g| id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@?MatroskaTags].) 0x19c-0x19f.7 (4)
| | | type: "master" 0x1a0-NA (0)
0x1a0|40 9d |@. | size: 157 0x1a0-0x1a1.7 (2)
| | | elements[0:3]: 0x1a2-0x23e.7 (157)
@ -462,7 +462,7 @@ $ fq -d matroska dv flac.mkv
0x4c0| 81 | . | size: 1 0x4c6-0x4c6.7 (1)
0x4c0| 01 | . | value: 1 0x4c7-0x4c7.7 (1)
| | | [1]{}: element 0x4c8-0x4cb.7 (4)
0x4c0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position of the Cluster containing the associated Block.) 0x4c8-0x4c8.7 (1)
0x4c0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position ((#segment-position)) of the Cluster containing the associated Block.) 0x4c8-0x4c8.7 (1)
| | | type: "uinteger" 0x4c9-NA (0)
0x4c0| 82 | . | size: 2 0x4c9-0x4c9.7 (1)
0x4c0| 02 0b | .. | value: 523 0x4ca-0x4cb.7 (2)

29
format/matroska/testdata/help_matroska.fqtest vendored Normal file
View File

@ -0,0 +1,29 @@
$ fq -h matroska
matroska: Matroska file decoder
Decode examples
===============
# Decode file as matroska
$ fq -d matroska . file
# Decode value as matroska
... | matroska
Lookup element using path
=========================
$ fq 'matroska_path(".Segment.Tracks[0)")' file.mkv
Get path to element
===================
$ fq 'grep_by(.id == "Tracks") | matroska_path' file.mkv
References
==========
- https://tools.ietf.org/html/draft-ietf-cellar-ebml-00
- https://matroska.org/technical/specs/index.html
- https://www.matroska.org/technical/basics.html
- https://www.matroska.org/technical/codec_specs.html
- https://wiki.xiph.org/MatroskaOpus

28
format/matroska/testdata/hevc.fqtest vendored
View File

@ -49,7 +49,7 @@ $ fq -d matroska dv hevc.mkv
0x0030|00 00 13 b7 |.... |
| | | elements[0:7]: 0x34-0x13ea.7 (5047)
| | | [0]{}: element 0x34-0x78.7 (69)
0x0030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains the Segment Position of other Top-Level Elements.) 0x34-0x37.7 (4)
0x0030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains seeking information of Top-Level Elements; see (#data-layout).) 0x34-0x37.7 (4)
| | | type: "master" 0x38-NA (0)
0x0030| c0 | . | size: 64 0x38-0x38.7 (1)
| | | elements[0:5]: 0x39-0x78.7 (64)
@ -65,12 +65,12 @@ $ fq -d matroska dv hevc.mkv
0x0040| 8b | . | size: 11 0x41-0x41.7 (1)
| | | elements[0:2]: 0x42-0x4c.7 (11)
| | | [0]{}: element 0x42-0x48.7 (7)
0x0040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x42-0x43.7 (2)
0x0040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x42-0x43.7 (2)
| | | type: "binary" 0x44-NA (0)
0x0040| 84 | . | size: 4 0x44-0x44.7 (1)
0x0040| 15 49 a9 66 | .I.f | value: raw bits 0x45-0x48.7 (4)
| | | [1]{}: element 0x49-0x4c.7 (4)
0x0040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x49-0x4a.7 (2)
0x0040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x49-0x4a.7 (2)
| | | type: "uinteger" 0x4b-NA (0)
0x0040| 81 | . | size: 1 0x4b-0x4b.7 (1)
0x0040| a1 | . | value: 161 0x4c-0x4c.7 (1)
@ -80,12 +80,12 @@ $ fq -d matroska dv hevc.mkv
0x0040| 8b| .| size: 11 0x4f-0x4f.7 (1)
| | | elements[0:2]: 0x50-0x5a.7 (11)
| | | [0]{}: element 0x50-0x56.7 (7)
0x0050|53 ab |S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x50-0x51.7 (2)
0x0050|53 ab |S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x50-0x51.7 (2)
| | | type: "binary" 0x52-NA (0)
0x0050| 84 | . | size: 4 0x52-0x52.7 (1)
0x0050| 16 54 ae 6b | .T.k | value: raw bits 0x53-0x56.7 (4)
| | | [1]{}: element 0x57-0x5a.7 (4)
0x0050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x57-0x58.7 (2)
0x0050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x57-0x58.7 (2)
| | | type: "uinteger" 0x59-NA (0)
0x0050| 81 | . | size: 1 0x59-0x59.7 (1)
0x0050| f1 | . | value: 241 0x5a-0x5a.7 (1)
@ -95,12 +95,12 @@ $ fq -d matroska dv hevc.mkv
0x0050| 8c | . | size: 12 0x5d-0x5d.7 (1)
| | | elements[0:2]: 0x5e-0x69.7 (12)
| | | [0]{}: element 0x5e-0x64.7 (7)
0x0050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x5e-0x5f.7 (2)
0x0050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x5e-0x5f.7 (2)
| | | type: "binary" 0x60-NA (0)
0x0060|84 |. | size: 4 0x60-0x60.7 (1)
0x0060| 12 54 c3 67 | .T.g | value: raw bits 0x61-0x64.7 (4)
| | | [1]{}: element 0x65-0x69.7 (5)
0x0060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x65-0x66.7 (2)
0x0060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x65-0x66.7 (2)
| | | type: "uinteger" 0x67-NA (0)
0x0060| 82 | . | size: 2 0x67-0x67.7 (1)
0x0060| 0a 8a | .. | value: 2698 0x68-0x69.7 (2)
@ -110,12 +110,12 @@ $ fq -d matroska dv hevc.mkv
0x0060| 8c | . | size: 12 0x6c-0x6c.7 (1)
| | | elements[0:2]: 0x6d-0x78.7 (12)
| | | [0]{}: element 0x6d-0x73.7 (7)
0x0060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x6d-0x6e.7 (2)
0x0060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x6d-0x6e.7 (2)
| | | type: "binary" 0x6f-NA (0)
0x0060| 84| .| size: 4 0x6f-0x6f.7 (1)
0x0070|1c 53 bb 6b |.S.k | value: raw bits 0x70-0x73.7 (4)
| | | [1]{}: element 0x74-0x78.7 (5)
0x0070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x74-0x75.7 (2)
0x0070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x74-0x75.7 (2)
| | | type: "uinteger" 0x76-NA (0)
0x0070| 82 | . | size: 2 0x76-0x76.7 (1)
0x0070| 13 9b | .. | value: 5019 0x77-0x78.7 (2)
@ -155,7 +155,7 @@ $ fq -d matroska dv hevc.mkv
0x00f0| 4c 61 76 66 35 38| Lavf58| value: "Lavf58.45.100" 0xfa-0x106.7 (13)
0x0100|2e 34 35 2e 31 30 30 |.45.100 |
| | | [4]{}: element 0x107-0x119.7 (19)
0x0100| 73 a4 | s. | id: "segment_uid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits).) 0x107-0x108.7 (2)
0x0100| 73 a4 | s. | id: "segment_uuid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits). It is effectively a Universally Unique IDentifier stored in binary form [@!RFC4122].) 0x107-0x108.7 (2)
| | | type: "binary" 0x109-NA (0)
0x0100| 90 | . | size: 16 0x109-0x109.7 (1)
0x0100| 64 a4 6f b2 71 ea| d.o.q.| value: raw bits 0x10a-0x119.7 (16)
@ -198,13 +198,13 @@ $ fq -d matroska dv hevc.mkv
0x0140| 81 | . | size: 1 0x149-0x149.7 (1)
0x0140| 00 | . | value: 0 0x14a-0x14a.7 (1)
| | | [3]{}: element 0x14b-0x151.7 (7)
0x0140| 22 b5 9c | ".. | id: "language" (0x22b59c) (Specifies the language of the track in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageIETF Element is used in the same TrackEntry.) 0x14b-0x14d.7 (3)
0x0140| 22 b5 9c | ".. | id: "language" (0x22b59c) (The language of the track, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageBCP47 Element is used in the same TrackEntry.) 0x14b-0x14d.7 (3)
| | | type: "string" 0x14e-NA (0)
0x0140| 83 | . | size: 3 0x14e-0x14e.7 (1)
0x0140| 75| u| value: "und" 0x14f-0x151.7 (3)
0x0150|6e 64 |nd |
| | | [4]{}: element 0x152-0x163.7 (18)
0x0150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@!MatroskaCodec] for more info.) 0x152-0x152.7 (1)
0x0150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@?MatroskaCodec] for more info.) 0x152-0x152.7 (1)
| | | type: "string" 0x153-NA (0)
0x0150| 90 | . | size: 16 0x153-0x153.7 (1)
0x0150| 56 5f 4d 50 45 47 48 2f 49 53 4f 2f| V_MPEGH/ISO/| value: "V_MPEGH/ISO/HEVC" 0x154-0x163.7 (16)
@ -547,7 +547,7 @@ $ fq -d matroska dv hevc.mkv
0x0200|ff ff ff c1 2c a2 de 09 b5 17 47 db bb 55 a4 fe|....,.....G..U..|
* |until 0xabd.7 (2244) | |
| | | [4]{}: element 0xabe-0xb63.7 (166)
0x0ab0| 12 54| .T| id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@!MatroskaTags].) 0xabe-0xac1.7 (4)
0x0ab0| 12 54| .T| id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@?MatroskaTags].) 0xabe-0xac1.7 (4)
0x0ac0|c3 67 |.g |
| | | type: "master" 0xac2-NA (0)
0x0ac0| 40 a0 | @. | size: 160 0xac2-0xac3.7 (2)
@ -704,7 +704,7 @@ $ fq -d matroska dv hevc.mkv
0x13e0| 81 | . | size: 1 0x13e2-0x13e2.7 (1)
0x13e0| 01 | . | value: 1 0x13e3-0x13e3.7 (1)
| | | [1]{}: element 0x13e4-0x13e7.7 (4)
0x13e0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position of the Cluster containing the associated Block.) 0x13e4-0x13e4.7 (1)
0x13e0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position ((#segment-position)) of the Cluster containing the associated Block.) 0x13e4-0x13e4.7 (1)
| | | type: "uinteger" 0x13e5-NA (0)
0x13e0| 82 | . | size: 2 0x13e5-0x13e5.7 (1)
0x13e0| 0b 30 | .0 | value: 2864 0x13e6-0x13e7.7 (2)

28
format/matroska/testdata/mp3.fqtest vendored
View File

@ -49,7 +49,7 @@ $ fq -d matroska dv mp3.mkv
0x030|00 00 04 a8 |.... |
| | | elements[0:7]: 0x34-0x4db.7 (1192)
| | | [0]{}: element 0x34-0x78.7 (69)
0x030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains the Segment Position of other Top-Level Elements.) 0x34-0x37.7 (4)
0x030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains seeking information of Top-Level Elements; see (#data-layout).) 0x34-0x37.7 (4)
| | | type: "master" 0x38-NA (0)
0x030| c0 | . | size: 64 0x38-0x38.7 (1)
| | | elements[0:5]: 0x39-0x78.7 (64)
@ -65,12 +65,12 @@ $ fq -d matroska dv mp3.mkv
0x040| 8b | . | size: 11 0x41-0x41.7 (1)
| | | elements[0:2]: 0x42-0x4c.7 (11)
| | | [0]{}: element 0x42-0x48.7 (7)
0x040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x42-0x43.7 (2)
0x040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x42-0x43.7 (2)
| | | type: "binary" 0x44-NA (0)
0x040| 84 | . | size: 4 0x44-0x44.7 (1)
0x040| 15 49 a9 66 | .I.f | value: raw bits 0x45-0x48.7 (4)
| | | [1]{}: element 0x49-0x4c.7 (4)
0x040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x49-0x4a.7 (2)
0x040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x49-0x4a.7 (2)
| | | type: "uinteger" 0x4b-NA (0)
0x040| 81 | . | size: 1 0x4b-0x4b.7 (1)
0x040| a1 | . | value: 161 0x4c-0x4c.7 (1)
@ -80,12 +80,12 @@ $ fq -d matroska dv mp3.mkv
0x040| 8b| .| size: 11 0x4f-0x4f.7 (1)
| | | elements[0:2]: 0x50-0x5a.7 (11)
| | | [0]{}: element 0x50-0x56.7 (7)
0x050|53 ab |S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x50-0x51.7 (2)
0x050|53 ab |S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x50-0x51.7 (2)
| | | type: "binary" 0x52-NA (0)
0x050| 84 | . | size: 4 0x52-0x52.7 (1)
0x050| 16 54 ae 6b | .T.k | value: raw bits 0x53-0x56.7 (4)
| | | [1]{}: element 0x57-0x5a.7 (4)
0x050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x57-0x58.7 (2)
0x050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x57-0x58.7 (2)
| | | type: "uinteger" 0x59-NA (0)
0x050| 81 | . | size: 1 0x59-0x59.7 (1)
0x050| f1 | . | value: 241 0x5a-0x5a.7 (1)
@ -95,12 +95,12 @@ $ fq -d matroska dv mp3.mkv
0x050| 8c | . | size: 12 0x5d-0x5d.7 (1)
| | | elements[0:2]: 0x5e-0x69.7 (12)
| | | [0]{}: element 0x5e-0x64.7 (7)
0x050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x5e-0x5f.7 (2)
0x050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x5e-0x5f.7 (2)
| | | type: "binary" 0x60-NA (0)
0x060|84 |. | size: 4 0x60-0x60.7 (1)
0x060| 12 54 c3 67 | .T.g | value: raw bits 0x61-0x64.7 (4)
| | | [1]{}: element 0x65-0x69.7 (5)
0x060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x65-0x66.7 (2)
0x060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x65-0x66.7 (2)
| | | type: "uinteger" 0x67-NA (0)
0x060| 82 | . | size: 2 0x67-0x67.7 (1)
0x060| 01 3e | .> | value: 318 0x68-0x69.7 (2)
@ -110,12 +110,12 @@ $ fq -d matroska dv mp3.mkv
0x060| 8c | . | size: 12 0x6c-0x6c.7 (1)
| | | elements[0:2]: 0x6d-0x78.7 (12)
| | | [0]{}: element 0x6d-0x73.7 (7)
0x060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x6d-0x6e.7 (2)
0x060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x6d-0x6e.7 (2)
| | | type: "binary" 0x6f-NA (0)
0x060| 84| .| size: 4 0x6f-0x6f.7 (1)
0x070|1c 53 bb 6b |.S.k | value: raw bits 0x70-0x73.7 (4)
| | | [1]{}: element 0x74-0x78.7 (5)
0x070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x74-0x75.7 (2)
0x070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x74-0x75.7 (2)
| | | type: "uinteger" 0x76-NA (0)
0x070| 82 | . | size: 2 0x76-0x76.7 (1)
0x070| 04 8c | .. | value: 1164 0x77-0x78.7 (2)
@ -155,7 +155,7 @@ $ fq -d matroska dv mp3.mkv
0x0f0| 4c 61 76 66 35 38| Lavf58| value: "Lavf58.45.100" 0xfa-0x106.7 (13)
0x100|2e 34 35 2e 31 30 30 |.45.100 |
| | | [4]{}: element 0x107-0x119.7 (19)
0x100| 73 a4 | s. | id: "segment_uid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits).) 0x107-0x108.7 (2)
0x100| 73 a4 | s. | id: "segment_uuid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits). It is effectively a Universally Unique IDentifier stored in binary form [@!RFC4122].) 0x107-0x108.7 (2)
| | | type: "binary" 0x109-NA (0)
0x100| 90 | . | size: 16 0x109-0x109.7 (1)
0x100| ed af 16 3e 3c c6| ...><.| value: raw bits 0x10a-0x119.7 (16)
@ -198,13 +198,13 @@ $ fq -d matroska dv mp3.mkv
0x140| 81 | . | size: 1 0x148-0x148.7 (1)
0x140| 00 | . | value: 0 0x149-0x149.7 (1)
| | | [3]{}: element 0x14a-0x150.7 (7)
0x140| 22 b5 9c | ".. | id: "language" (0x22b59c) (Specifies the language of the track in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageIETF Element is used in the same TrackEntry.) 0x14a-0x14c.7 (3)
0x140| 22 b5 9c | ".. | id: "language" (0x22b59c) (The language of the track, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageBCP47 Element is used in the same TrackEntry.) 0x14a-0x14c.7 (3)
| | | type: "string" 0x14d-NA (0)
0x140| 83 | . | size: 3 0x14d-0x14d.7 (1)
0x140| 75 6e| un| value: "und" 0x14e-0x150.7 (3)
0x150|64 |d |
| | | [4]{}: element 0x151-0x15b.7 (11)
0x150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@!MatroskaCodec] for more info.) 0x151-0x151.7 (1)
0x150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@?MatroskaCodec] for more info.) 0x151-0x151.7 (1)
| | | type: "string" 0x152-NA (0)
0x150| 89 | . | size: 9 0x152-0x152.7 (1)
0x150| 41 5f 4d 50 45 47 2f 4c 33 | A_MPEG/L3 | value: "A_MPEG/L3" 0x153-0x15b.7 (9)
@ -234,7 +234,7 @@ $ fq -d matroska dv mp3.mkv
0x170|81 |. | size: 1 0x170-0x170.7 (1)
0x170| 10 | . | value: 16 0x171-0x171.7 (1)
| | | [4]{}: element 0x172-0x21a.7 (169)
0x170| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@!MatroskaTags].) 0x172-0x175.7 (4)
0x170| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@?MatroskaTags].) 0x172-0x175.7 (4)
| | | type: "master" 0x176-NA (0)
0x170| 40 a3 | @. | size: 163 0x176-0x177.7 (2)
| | | elements[0:3]: 0x178-0x21a.7 (163)
@ -595,7 +595,7 @@ $ fq -d matroska dv mp3.mkv
0x4d0| 81 | . | size: 1 0x4d3-0x4d3.7 (1)
0x4d0| 01 | . | value: 1 0x4d4-0x4d4.7 (1)
| | | [1]{}: element 0x4d5-0x4d8.7 (4)
0x4d0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position of the Cluster containing the associated Block.) 0x4d5-0x4d5.7 (1)
0x4d0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position ((#segment-position)) of the Cluster containing the associated Block.) 0x4d5-0x4d5.7 (1)
| | | type: "uinteger" 0x4d6-NA (0)
0x4d0| 82 | . | size: 2 0x4d6-0x4d6.7 (1)
0x4d0| 01 e7 | .. | value: 487 0x4d7-0x4d8.7 (2)

28
format/matroska/testdata/mpeg2.fqtest vendored
View File

@ -49,7 +49,7 @@ $ fq -d matroska dv mpeg2.mkv
0x0030|00 00 21 96 |..!. |
| | | elements[0:7]: 0x34-0x21c9.7 (8598)
| | | [0]{}: element 0x34-0x78.7 (69)
0x0030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains the Segment Position of other Top-Level Elements.) 0x34-0x37.7 (4)
0x0030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains seeking information of Top-Level Elements; see (#data-layout).) 0x34-0x37.7 (4)
| | | type: "master" 0x38-NA (0)
0x0030| c0 | . | size: 64 0x38-0x38.7 (1)
| | | elements[0:5]: 0x39-0x78.7 (64)
@ -65,12 +65,12 @@ $ fq -d matroska dv mpeg2.mkv
0x0040| 8b | . | size: 11 0x41-0x41.7 (1)
| | | elements[0:2]: 0x42-0x4c.7 (11)
| | | [0]{}: element 0x42-0x48.7 (7)
0x0040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x42-0x43.7 (2)
0x0040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x42-0x43.7 (2)
| | | type: "binary" 0x44-NA (0)
0x0040| 84 | . | size: 4 0x44-0x44.7 (1)
0x0040| 15 49 a9 66 | .I.f | value: raw bits 0x45-0x48.7 (4)
| | | [1]{}: element 0x49-0x4c.7 (4)
0x0040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x49-0x4a.7 (2)
0x0040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x49-0x4a.7 (2)
| | | type: "uinteger" 0x4b-NA (0)
0x0040| 81 | . | size: 1 0x4b-0x4b.7 (1)
0x0040| a1 | . | value: 161 0x4c-0x4c.7 (1)
@ -80,12 +80,12 @@ $ fq -d matroska dv mpeg2.mkv
0x0040| 8b| .| size: 11 0x4f-0x4f.7 (1)
| | | elements[0:2]: 0x50-0x5a.7 (11)
| | | [0]{}: element 0x50-0x56.7 (7)
0x0050|53 ab |S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x50-0x51.7 (2)
0x0050|53 ab |S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x50-0x51.7 (2)
| | | type: "binary" 0x52-NA (0)
0x0050| 84 | . | size: 4 0x52-0x52.7 (1)
0x0050| 16 54 ae 6b | .T.k | value: raw bits 0x53-0x56.7 (4)
| | | [1]{}: element 0x57-0x5a.7 (4)
0x0050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x57-0x58.7 (2)
0x0050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x57-0x58.7 (2)
| | | type: "uinteger" 0x59-NA (0)
0x0050| 81 | . | size: 1 0x59-0x59.7 (1)
0x0050| f1 | . | value: 241 0x5a-0x5a.7 (1)
@ -95,12 +95,12 @@ $ fq -d matroska dv mpeg2.mkv
0x0050| 8c | . | size: 12 0x5d-0x5d.7 (1)
| | | elements[0:2]: 0x5e-0x69.7 (12)
| | | [0]{}: element 0x5e-0x64.7 (7)
0x0050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x5e-0x5f.7 (2)
0x0050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x5e-0x5f.7 (2)
| | | type: "binary" 0x60-NA (0)
0x0060|84 |. | size: 4 0x60-0x60.7 (1)
0x0060| 12 54 c3 67 | .T.g | value: raw bits 0x61-0x64.7 (4)
| | | [1]{}: element 0x65-0x69.7 (5)
0x0060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x65-0x66.7 (2)
0x0060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x65-0x66.7 (2)
| | | type: "uinteger" 0x67-NA (0)
0x0060| 82 | . | size: 2 0x67-0x67.7 (1)
0x0060| 01 41 | .A | value: 321 0x68-0x69.7 (2)
@ -110,12 +110,12 @@ $ fq -d matroska dv mpeg2.mkv
0x0060| 8c | . | size: 12 0x6c-0x6c.7 (1)
| | | elements[0:2]: 0x6d-0x78.7 (12)
| | | [0]{}: element 0x6d-0x73.7 (7)
0x0060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x6d-0x6e.7 (2)
0x0060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x6d-0x6e.7 (2)
| | | type: "binary" 0x6f-NA (0)
0x0060| 84| .| size: 4 0x6f-0x6f.7 (1)
0x0070|1c 53 bb 6b |.S.k | value: raw bits 0x70-0x73.7 (4)
| | | [1]{}: element 0x74-0x78.7 (5)
0x0070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x74-0x75.7 (2)
0x0070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x74-0x75.7 (2)
| | | type: "uinteger" 0x76-NA (0)
0x0070| 82 | . | size: 2 0x76-0x76.7 (1)
0x0070| 21 7a | !z | value: 8570 0x77-0x78.7 (2)
@ -155,7 +155,7 @@ $ fq -d matroska dv mpeg2.mkv
0x00f0| 4c 61 76 66 35 38| Lavf58| value: "Lavf58.45.100" 0xfa-0x106.7 (13)
0x0100|2e 34 35 2e 31 30 30 |.45.100 |
| | | [4]{}: element 0x107-0x119.7 (19)
0x0100| 73 a4 | s. | id: "segment_uid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits).) 0x107-0x108.7 (2)
0x0100| 73 a4 | s. | id: "segment_uuid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits). It is effectively a Universally Unique IDentifier stored in binary form [@!RFC4122].) 0x107-0x108.7 (2)
| | | type: "binary" 0x109-NA (0)
0x0100| 90 | . | size: 16 0x109-0x109.7 (1)
0x0100| 97 68 53 5b 2a ea| .hS[*.| value: raw bits 0x10a-0x119.7 (16)
@ -198,13 +198,13 @@ $ fq -d matroska dv mpeg2.mkv
0x0140| 81 | . | size: 1 0x148-0x148.7 (1)
0x0140| 00 | . | value: 0 0x149-0x149.7 (1)
| | | [3]{}: element 0x14a-0x150.7 (7)
0x0140| 22 b5 9c | ".. | id: "language" (0x22b59c) (Specifies the language of the track in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageIETF Element is used in the same TrackEntry.) 0x14a-0x14c.7 (3)
0x0140| 22 b5 9c | ".. | id: "language" (0x22b59c) (The language of the track, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageBCP47 Element is used in the same TrackEntry.) 0x14a-0x14c.7 (3)
| | | type: "string" 0x14d-NA (0)
0x0140| 83 | . | size: 3 0x14d-0x14d.7 (1)
0x0140| 75 6e| un| value: "und" 0x14e-0x150.7 (3)
0x0150|64 |d |
| | | [4]{}: element 0x151-0x159.7 (9)
0x0150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@!MatroskaCodec] for more info.) 0x151-0x151.7 (1)
0x0150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@?MatroskaCodec] for more info.) 0x151-0x151.7 (1)
| | | type: "string" 0x152-NA (0)
0x0150| 87 | . | size: 7 0x152-0x152.7 (1)
0x0150| 56 5f 4d 50 45 47 32 | V_MPEG2 | value: "V_MPEG2" 0x153-0x159.7 (7)
@ -234,7 +234,7 @@ $ fq -d matroska dv mpeg2.mkv
0x0170| 81 | . | size: 1 0x173-0x173.7 (1)
0x0170| f0 | . | value: 240 0x174-0x174.7 (1)
| | | [4]{}: element 0x175-0x21d.7 (169)
0x0170| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@!MatroskaTags].) 0x175-0x178.7 (4)
0x0170| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@?MatroskaTags].) 0x175-0x178.7 (4)
| | | type: "master" 0x179-NA (0)
0x0170| 40 a3 | @. | size: 163 0x179-0x17a.7 (2)
| | | elements[0:3]: 0x17b-0x21d.7 (163)
@ -396,7 +396,7 @@ $ fq -d matroska dv mpeg2.mkv
0x21c0| 81 | . | size: 1 0x21c1-0x21c1.7 (1)
0x21c0| 01 | . | value: 1 0x21c2-0x21c2.7 (1)
| | | [1]{}: element 0x21c3-0x21c6.7 (4)
0x21c0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position of the Cluster containing the associated Block.) 0x21c3-0x21c3.7 (1)
0x21c0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position ((#segment-position)) of the Cluster containing the associated Block.) 0x21c3-0x21c3.7 (1)
| | | type: "uinteger" 0x21c4-NA (0)
0x21c0| 82 | . | size: 2 0x21c4-0x21c4.7 (1)
0x21c0| 01 ea | .. | value: 490 0x21c5-0x21c6.7 (2)

28
format/matroska/testdata/opus.fqtest vendored
View File

@ -49,7 +49,7 @@ $ fq -d matroska dv opus.mkv
0x030|00 00 03 b9 |.... |
| | | elements[0:7]: 0x34-0x3ec.7 (953)
| | | [0]{}: element 0x34-0x78.7 (69)
0x030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains the Segment Position of other Top-Level Elements.) 0x34-0x37.7 (4)
0x030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains seeking information of Top-Level Elements; see (#data-layout).) 0x34-0x37.7 (4)
| | | type: "master" 0x38-NA (0)
0x030| c0 | . | size: 64 0x38-0x38.7 (1)
| | | elements[0:5]: 0x39-0x78.7 (64)
@ -65,12 +65,12 @@ $ fq -d matroska dv opus.mkv
0x040| 8b | . | size: 11 0x41-0x41.7 (1)
| | | elements[0:2]: 0x42-0x4c.7 (11)
| | | [0]{}: element 0x42-0x48.7 (7)
0x040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x42-0x43.7 (2)
0x040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x42-0x43.7 (2)
| | | type: "binary" 0x44-NA (0)
0x040| 84 | . | size: 4 0x44-0x44.7 (1)
0x040| 15 49 a9 66 | .I.f | value: raw bits 0x45-0x48.7 (4)
| | | [1]{}: element 0x49-0x4c.7 (4)
0x040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x49-0x4a.7 (2)
0x040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x49-0x4a.7 (2)
| | | type: "uinteger" 0x4b-NA (0)
0x040| 81 | . | size: 1 0x4b-0x4b.7 (1)
0x040| a1 | . | value: 161 0x4c-0x4c.7 (1)
@ -80,12 +80,12 @@ $ fq -d matroska dv opus.mkv
0x040| 8b| .| size: 11 0x4f-0x4f.7 (1)
| | | elements[0:2]: 0x50-0x5a.7 (11)
| | | [0]{}: element 0x50-0x56.7 (7)
0x050|53 ab |S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x50-0x51.7 (2)
0x050|53 ab |S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x50-0x51.7 (2)
| | | type: "binary" 0x52-NA (0)
0x050| 84 | . | size: 4 0x52-0x52.7 (1)
0x050| 16 54 ae 6b | .T.k | value: raw bits 0x53-0x56.7 (4)
| | | [1]{}: element 0x57-0x5a.7 (4)
0x050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x57-0x58.7 (2)
0x050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x57-0x58.7 (2)
| | | type: "uinteger" 0x59-NA (0)
0x050| 81 | . | size: 1 0x59-0x59.7 (1)
0x050| f1 | . | value: 241 0x5a-0x5a.7 (1)
@ -95,12 +95,12 @@ $ fq -d matroska dv opus.mkv
0x050| 8c | . | size: 12 0x5d-0x5d.7 (1)
| | | elements[0:2]: 0x5e-0x69.7 (12)
| | | [0]{}: element 0x5e-0x64.7 (7)
0x050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x5e-0x5f.7 (2)
0x050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x5e-0x5f.7 (2)
| | | type: "binary" 0x60-NA (0)
0x060|84 |. | size: 4 0x60-0x60.7 (1)
0x060| 12 54 c3 67 | .T.g | value: raw bits 0x61-0x64.7 (4)
| | | [1]{}: element 0x65-0x69.7 (5)
0x060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x65-0x66.7 (2)
0x060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x65-0x66.7 (2)
| | | type: "uinteger" 0x67-NA (0)
0x060| 82 | . | size: 2 0x67-0x67.7 (1)
0x060| 01 5e | .^ | value: 350 0x68-0x69.7 (2)
@ -110,12 +110,12 @@ $ fq -d matroska dv opus.mkv
0x060| 8c | . | size: 12 0x6c-0x6c.7 (1)
| | | elements[0:2]: 0x6d-0x78.7 (12)
| | | [0]{}: element 0x6d-0x73.7 (7)
0x060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x6d-0x6e.7 (2)
0x060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x6d-0x6e.7 (2)
| | | type: "binary" 0x6f-NA (0)
0x060| 84| .| size: 4 0x6f-0x6f.7 (1)
0x070|1c 53 bb 6b |.S.k | value: raw bits 0x70-0x73.7 (4)
| | | [1]{}: element 0x74-0x78.7 (5)
0x070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x74-0x75.7 (2)
0x070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x74-0x75.7 (2)
| | | type: "uinteger" 0x76-NA (0)
0x070| 82 | . | size: 2 0x76-0x76.7 (1)
0x070| 03 9d | .. | value: 925 0x77-0x78.7 (2)
@ -155,7 +155,7 @@ $ fq -d matroska dv opus.mkv
0x0f0| 4c 61 76 66 35 38| Lavf58| value: "Lavf58.45.100" 0xfa-0x106.7 (13)
0x100|2e 34 35 2e 31 30 30 |.45.100 |
| | | [4]{}: element 0x107-0x119.7 (19)
0x100| 73 a4 | s. | id: "segment_uid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits).) 0x107-0x108.7 (2)
0x100| 73 a4 | s. | id: "segment_uuid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits). It is effectively a Universally Unique IDentifier stored in binary form [@!RFC4122].) 0x107-0x108.7 (2)
| | | type: "binary" 0x109-NA (0)
0x100| 90 | . | size: 16 0x109-0x109.7 (1)
0x100| a0 33 a6 65 a7 8d| .3.e..| value: raw bits 0x10a-0x119.7 (16)
@ -198,13 +198,13 @@ $ fq -d matroska dv opus.mkv
0x140| 81 | . | size: 1 0x148-0x148.7 (1)
0x140| 00 | . | value: 0 0x149-0x149.7 (1)
| | | [3]{}: element 0x14a-0x150.7 (7)
0x140| 22 b5 9c | ".. | id: "language" (0x22b59c) (Specifies the language of the track in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageIETF Element is used in the same TrackEntry.) 0x14a-0x14c.7 (3)
0x140| 22 b5 9c | ".. | id: "language" (0x22b59c) (The language of the track, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageBCP47 Element is used in the same TrackEntry.) 0x14a-0x14c.7 (3)
| | | type: "string" 0x14d-NA (0)
0x140| 83 | . | size: 3 0x14d-0x14d.7 (1)
0x140| 75 6e| un| value: "und" 0x14e-0x150.7 (3)
0x150|64 |d |
| | | [4]{}: element 0x151-0x158.7 (8)
0x150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@!MatroskaCodec] for more info.) 0x151-0x151.7 (1)
0x150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@?MatroskaCodec] for more info.) 0x151-0x151.7 (1)
| | | type: "string" 0x152-NA (0)
0x150| 86 | . | size: 6 0x152-0x152.7 (1)
0x150| 41 5f 4f 50 55 53 | A_OPUS | value: "A_OPUS" 0x153-0x158.7 (6)
@ -260,7 +260,7 @@ $ fq -d matroska dv opus.mkv
0x190|00 |. |
0x190| 00 | . | map_family: 0 0x191-0x191.7 (1)
| | | [4]{}: element 0x192-0x234.7 (163)
0x190| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@!MatroskaTags].) 0x192-0x195.7 (4)
0x190| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@?MatroskaTags].) 0x192-0x195.7 (4)
| | | type: "master" 0x196-NA (0)
0x190| 40 9d | @. | size: 157 0x196-0x197.7 (2)
| | | elements[0:3]: 0x198-0x234.7 (157)
@ -487,7 +487,7 @@ $ fq -d matroska dv opus.mkv
0x3e0| 81 | . | size: 1 0x3e4-0x3e4.7 (1)
0x3e0| 01 | . | value: 1 0x3e5-0x3e5.7 (1)
| | | [1]{}: element 0x3e6-0x3e9.7 (4)
0x3e0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position of the Cluster containing the associated Block.) 0x3e6-0x3e6.7 (1)
0x3e0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position ((#segment-position)) of the Cluster containing the associated Block.) 0x3e6-0x3e6.7 (1)
| | | type: "uinteger" 0x3e7-NA (0)
0x3e0| 82 | . | size: 2 0x3e7-0x3e7.7 (1)
0x3e0| 02 01 | .. | value: 513 0x3e8-0x3e9.7 (2)

70
format/matroska/testdata/sweep-with-DC.mkvmerge13.mka.fqtest vendored
View File

@ -51,7 +51,7 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x0030|00 00 48 4b |..HK |
| | | elements[0:11]: 0x34-0x487e.7 (18507)
| | | [0]{}: element 0x34-0x74.7 (65)
0x0030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains the Segment Position of other Top-Level Elements.) 0x34-0x37.7 (4)
0x0030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains seeking information of Top-Level Elements; see (#data-layout).) 0x34-0x37.7 (4)
| | | type: "master" 0x38-NA (0)
0x0030| bc | . | size: 60 0x38-0x38.7 (1)
| | | elements[0:4]: 0x39-0x74.7 (60)
@ -61,13 +61,13 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x0030| 8c | . | size: 12 0x3b-0x3b.7 (1)
| | | elements[0:2]: 0x3c-0x47.7 (12)
| | | [0]{}: element 0x3c-0x42.7 (7)
0x0030| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x3c-0x3d.7 (2)
0x0030| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x3c-0x3d.7 (2)
| | | type: "binary" 0x3e-NA (0)
0x0030| 84 | . | size: 4 0x3e-0x3e.7 (1)
0x0030| 15| .| value: raw bits 0x3f-0x42.7 (4)
0x0040|49 a9 66 |I.f |
| | | [1]{}: element 0x43-0x47.7 (5)
0x0040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x43-0x44.7 (2)
0x0040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x43-0x44.7 (2)
| | | type: "uinteger" 0x45-NA (0)
0x0040| 82 | . | size: 2 0x45-0x45.7 (1)
0x0040| 10 03 | .. | value: 4099 0x46-0x47.7 (2)
@ -77,13 +77,13 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x0040| 8c | . | size: 12 0x4a-0x4a.7 (1)
| | | elements[0:2]: 0x4b-0x56.7 (12)
| | | [0]{}: element 0x4b-0x51.7 (7)
0x0040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x4b-0x4c.7 (2)
0x0040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x4b-0x4c.7 (2)
| | | type: "binary" 0x4d-NA (0)
0x0040| 84 | . | size: 4 0x4d-0x4d.7 (1)
0x0040| 16 54| .T| value: raw bits 0x4e-0x51.7 (4)
0x0050|ae 6b |.k |
| | | [1]{}: element 0x52-0x56.7 (5)
0x0050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x52-0x53.7 (2)
0x0050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x52-0x53.7 (2)
| | | type: "uinteger" 0x54-NA (0)
0x0050| 82 | . | size: 2 0x54-0x54.7 (1)
0x0050| 10 8f | .. | value: 4239 0x55-0x56.7 (2)
@ -93,13 +93,13 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x0050| 8c | . | size: 12 0x59-0x59.7 (1)
| | | elements[0:2]: 0x5a-0x65.7 (12)
| | | [0]{}: element 0x5a-0x60.7 (7)
0x0050| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x5a-0x5b.7 (2)
0x0050| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x5a-0x5b.7 (2)
| | | type: "binary" 0x5c-NA (0)
0x0050| 84 | . | size: 4 0x5c-0x5c.7 (1)
0x0050| 1c 53 bb| .S.| value: raw bits 0x5d-0x60.7 (4)
0x0060|6b |k |
| | | [1]{}: element 0x61-0x65.7 (5)
0x0060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x61-0x62.7 (2)
0x0060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x61-0x62.7 (2)
| | | type: "uinteger" 0x63-NA (0)
0x0060| 82 | . | size: 2 0x63-0x63.7 (1)
0x0060| 46 41 | FA | value: 17985 0x64-0x65.7 (2)
@ -109,12 +109,12 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x0060| 8c | . | size: 12 0x68-0x68.7 (1)
| | | elements[0:2]: 0x69-0x74.7 (12)
| | | [0]{}: element 0x69-0x6f.7 (7)
0x0060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x69-0x6a.7 (2)
0x0060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x69-0x6a.7 (2)
| | | type: "binary" 0x6b-NA (0)
0x0060| 84 | . | size: 4 0x6b-0x6b.7 (1)
0x0060| 12 54 c3 67| .T.g| value: raw bits 0x6c-0x6f.7 (4)
| | | [1]{}: element 0x70-0x74.7 (5)
0x0070|53 ac |S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x70-0x71.7 (2)
0x0070|53 ac |S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x70-0x71.7 (2)
| | | type: "uinteger" 0x72-NA (0)
0x0070| 82 | . | size: 2 0x72-0x72.7 (1)
0x0070| 46 a2 | F. | value: 18082 0x73-0x74.7 (2)
@ -161,7 +161,7 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x10a0| 88 | . | size: 8 0x10a7-0x10a7.7 (1)
0x10a0| 09 7b f0 a0 97 43 d2 00| .{...C..| value: raw bits 0x10a8-0x10af.7 (8)
| | | [5]{}: element 0x10b0-0x10c2.7 (19)
0x10b0|73 a4 |s. | id: "segment_uid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits).) 0x10b0-0x10b1.7 (2)
0x10b0|73 a4 |s. | id: "segment_uuid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits). It is effectively a Universally Unique IDentifier stored in binary form [@!RFC4122].) 0x10b0-0x10b1.7 (2)
| | | type: "binary" 0x10b2-NA (0)
0x10b0| 90 | . | size: 16 0x10b2-0x10b2.7 (1)
0x10b0| 3b 9f 28 c7 c4 90 8a e0 cd 66 8f 11 8f| ;.(......f...| value: raw bits 0x10b3-0x10c2.7 (16)
@ -193,7 +193,7 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x10d0| 81 | . | size: 1 0x10db-0x10db.7 (1)
0x10d0| 02 | . | value: "audio" (2) (Audio samples.) 0x10dc-0x10dc.7 (1)
| | | [3]{}: element 0x10dd-0x10e4.7 (8)
0x10d0| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@!MatroskaCodec] for more info.) 0x10dd-0x10dd.7 (1)
0x10d0| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@?MatroskaCodec] for more info.) 0x10dd-0x10dd.7 (1)
| | | type: "string" 0x10de-NA (0)
0x10d0| 86 | . | size: 6 0x10de-0x10de.7 (1)
0x10d0| 41| A| value: "A_FLAC" 0x10df-0x10e4.7 (6)
@ -254,12 +254,12 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x1170| 84 | . | size: 4 0x1171-0x1171.7 (1)
0x1170| 02 6c 0a 28 | .l.( | value: 40634920 0x1172-0x1175.7 (4)
| | | [6]{}: element 0x1176-0x117c.7 (7)
0x1170| 22 b5 9c | ".. | id: "language" (0x22b59c) (Specifies the language of the track in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageIETF Element is used in the same TrackEntry.) 0x1176-0x1178.7 (3)
0x1170| 22 b5 9c | ".. | id: "language" (0x22b59c) (The language of the track, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageBCP47 Element is used in the same TrackEntry.) 0x1176-0x1178.7 (3)
| | | type: "string" 0x1179-NA (0)
0x1170| 83 | . | size: 3 0x1179-0x1179.7 (1)
0x1170| 75 6e 64 | und | value: "und" 0x117a-0x117c.7 (3)
| | | [7]{}: element 0x117d-0x1183.7 (7)
0x1170| 22 b5 9d| "..| id: "language_ietf" (0x22b59d) (Specifies the language of the track according to [@!BCP47] and using the IANA Language Subtag Registry [@!IANALangRegistry]. If this Element is used, then any Language Elements used in the same TrackEntry **MUST** be ignored.) 0x117d-0x117f.7 (3)
0x1170| 22 b5 9d| "..| id: "language_bcp47" (0x22b59d) (The language of the track, in the [@!BCP47] form; see (#language-codes) on language codes. If this Element is used, then any Language Elements used in the same TrackEntry **MUST** be ignored.) 0x117d-0x117f.7 (3)
| | | type: "string" 0x1180-NA (0)
0x1180|83 |. | size: 3 0x1180-0x1180.7 (1)
0x1180| 75 6e 64 | und | value: "und" 0x1181-0x1183.7 (3)
@ -2957,7 +2957,7 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x4680| 81 | . | size: 1 0x4682-0x4682.7 (1)
0x4680| 01 | . | value: 1 0x4683-0x4683.7 (1)
| | | [1]{}: element 0x4684-0x4687.7 (4)
0x4680| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position of the Cluster containing the associated Block.) 0x4684-0x4684.7 (1)
0x4680| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position ((#segment-position)) of the Cluster containing the associated Block.) 0x4684-0x4684.7 (1)
| | | type: "uinteger" 0x4685-NA (0)
0x4680| 82 | . | size: 2 0x4685-0x4685.7 (1)
0x4680| 15 86 | .. | value: 5510 0x4686-0x4687.7 (2)
@ -2988,7 +2988,7 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x4690| 81 | . | size: 1 0x4694-0x4694.7 (1)
0x4690| 01 | . | value: 1 0x4695-0x4695.7 (1)
| | | [1]{}: element 0x4696-0x4699.7 (4)
0x4690| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position of the Cluster containing the associated Block.) 0x4696-0x4696.7 (1)
0x4690| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position ((#segment-position)) of the Cluster containing the associated Block.) 0x4696-0x4696.7 (1)
| | | type: "uinteger" 0x4697-NA (0)
0x4690| 82 | . | size: 2 0x4697-0x4697.7 (1)
0x4690| 15 86 | .. | value: 5510 0x4698-0x4699.7 (2)
@ -3018,7 +3018,7 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x46a0| 81 | . | size: 1 0x46a6-0x46a6.7 (1)
0x46a0| 01 | . | value: 1 0x46a7-0x46a7.7 (1)
| | | [1]{}: element 0x46a8-0x46ab.7 (4)
0x46a0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position of the Cluster containing the associated Block.) 0x46a8-0x46a8.7 (1)
0x46a0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position ((#segment-position)) of the Cluster containing the associated Block.) 0x46a8-0x46a8.7 (1)
| | | type: "uinteger" 0x46a9-NA (0)
0x46a0| 82 | . | size: 2 0x46a9-0x46a9.7 (1)
0x46a0| 19 46 | .F | value: 6470 0x46aa-0x46ab.7 (2)
@ -3048,7 +3048,7 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x46b0| 81 | . | size: 1 0x46b9-0x46b9.7 (1)
0x46b0| 01 | . | value: 1 0x46ba-0x46ba.7 (1)
| | | [1]{}: element 0x46bb-0x46be.7 (4)
0x46b0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position of the Cluster containing the associated Block.) 0x46bb-0x46bb.7 (1)
0x46b0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position ((#segment-position)) of the Cluster containing the associated Block.) 0x46bb-0x46bb.7 (1)
| | | type: "uinteger" 0x46bc-NA (0)
0x46b0| 82 | . | size: 2 0x46bc-0x46bc.7 (1)
0x46b0| 2b 6b | +k | value: 11115 0x46bd-0x46be.7 (2)
@ -3078,7 +3078,7 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x46c0| 81 | . | size: 1 0x46cc-0x46cc.7 (1)
0x46c0| 01 | . | value: 1 0x46cd-0x46cd.7 (1)
| | | [1]{}: element 0x46ce-0x46d1.7 (4)
0x46c0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position of the Cluster containing the associated Block.) 0x46ce-0x46ce.7 (1)
0x46c0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position ((#segment-position)) of the Cluster containing the associated Block.) 0x46ce-0x46ce.7 (1)
| | | type: "uinteger" 0x46cf-NA (0)
0x46c0| 82| .| size: 2 0x46cf-0x46cf.7 (1)
0x46d0|2b 6b |+k | value: 11115 0x46d0-0x46d1.7 (2)
@ -3088,7 +3088,7 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x46d0| 82 | . | size: 2 0x46d3-0x46d3.7 (1)
0x46d0| 07 d6 | .. | value: 2006 0x46d4-0x46d5.7 (2)
| | | [10]{}: element 0x46d6-0x487e.7 (425)
0x46d0| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@!MatroskaTags].) 0x46d6-0x46d9.7 (4)
0x46d0| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@?MatroskaTags].) 0x46d6-0x46d9.7 (4)
| | | type: "master" 0x46da-NA (0)
0x46d0| 41 a3 | A. | size: 419 0x46da-0x46db.7 (2)
| | | elements[0:1]: 0x46dc-0x487e.7 (419)
@ -3114,10 +3114,10 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x46e0| 34 a9 ce 93 aa 66| 4....f| value: 3794791294650729286 0x46ea-0x46f1.7 (8)
0x46f0|c3 46 |.F |
| | | [2]{}: element 0x46f2-0x46f9.7 (8)
0x46f0| 63 ca | c. | id: "target_type" (0x63ca) (An informational string that can be used to display the logical level of the target like "ALBUM", "TRACK", "MOVIE", "CHAPTER", etc ; see Section 6.4 of [@!MatroskaTags].) 0x46f2-0x46f3.7 (2)
0x46f0| 63 ca | c. | id: "target_type" (0x63ca) (An informational string that can be used to display the logical level of the target like "ALBUM", "TRACK", "MOVIE", "CHAPTER", etc ; see Section 6.4 of [@?MatroskaTags].) 0x46f2-0x46f3.7 (2)
| | | type: "string" 0x46f4-NA (0)
0x46f0| 85 | . | size: 5 0x46f4-0x46f4.7 (1)
0x46f0| 4d 4f 56 49 45 | MOVIE | value: "movie" ("MOVIE") 0x46f5-0x46f9.7 (5)
0x46f0| 4d 4f 56 49 45 | MOVIE | value: "targettypevalue_50" ("MOVIE") 0x46f5-0x46f9.7 (5)
| | | [1]{}: element 0x46fa-0x4715.7 (28)
0x46f0| 67 c8 | g. | id: "simple_tag" (0x67c8) (Contains general information about the target.) 0x46fa-0x46fb.7 (2)
| | | type: "master" 0x46fc-NA (0)
@ -3134,13 +3134,13 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x4700| 85 | . | size: 5 0x4705-0x4705.7 (1)
0x4700| 33 37 37 34 37 | 37747 | value: "37747" 0x4706-0x470a.7 (5)
| | | [2]{}: element 0x470b-0x4710.7 (6)
0x4700| 44 7a | Dz | id: "tag_language" (0x447a) (Specifies the language of the tag specified, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the TagLanguageIETF Element is used within the same SimpleTag Element.) 0x470b-0x470c.7 (2)
0x4700| 44 7a | Dz | id: "tag_language" (0x447a) (Specifies the language of the tag specified, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the TagLanguageBCP47 Element is used within the same SimpleTag Element.) 0x470b-0x470c.7 (2)
| | | type: "string" 0x470d-NA (0)
0x4700| 83 | . | size: 3 0x470d-0x470d.7 (1)
0x4700| 65 6e| en| value: "eng" 0x470e-0x4710.7 (3)
0x4710|67 |g |
| | | [3]{}: element 0x4711-0x4715.7 (5)
0x4710| 44 7b | D{ | id: "tag_language_ietf" (0x447b) (Specifies the language used in the TagString according to [@!BCP47] and using the IANA Language Subtag Registry [@!IANALangRegistry]. If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.) 0x4711-0x4712.7 (2)
0x4710| 44 7b | D{ | id: "tag_language_bcp47" (0x447b) (The language used in the TagString, in the [@!BCP47] form; see (#language-codes) on language codes. If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.) 0x4711-0x4712.7 (2)
| | | type: "string" 0x4713-NA (0)
0x4710| 82 | . | size: 2 0x4713-0x4713.7 (1)
0x4710| 65 6e | en | value: "en" 0x4714-0x4715.7 (2)
@ -3162,12 +3162,12 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x4720| 30 30 3a 30 30 3a 30 32 2e| 00:00:02.| value: "00:00:02.607714066" 0x4727-0x4738.7 (18)
0x4730|36 30 37 37 31 34 30 36 36 |607714066 |
| | | [2]{}: element 0x4739-0x473e.7 (6)
0x4730| 44 7a | Dz | id: "tag_language" (0x447a) (Specifies the language of the tag specified, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the TagLanguageIETF Element is used within the same SimpleTag Element.) 0x4739-0x473a.7 (2)
0x4730| 44 7a | Dz | id: "tag_language" (0x447a) (Specifies the language of the tag specified, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the TagLanguageBCP47 Element is used within the same SimpleTag Element.) 0x4739-0x473a.7 (2)
| | | type: "string" 0x473b-NA (0)
0x4730| 83 | . | size: 3 0x473b-0x473b.7 (1)
0x4730| 65 6e 67 | eng | value: "eng" 0x473c-0x473e.7 (3)
| | | [3]{}: element 0x473f-0x4743.7 (5)
0x4730| 44| D| id: "tag_language_ietf" (0x447b) (Specifies the language used in the TagString according to [@!BCP47] and using the IANA Language Subtag Registry [@!IANALangRegistry]. If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.) 0x473f-0x4740.7 (2)
0x4730| 44| D| id: "tag_language_bcp47" (0x447b) (The language used in the TagString, in the [@!BCP47] form; see (#language-codes) on language codes. If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.) 0x473f-0x4740.7 (2)
0x4740|7b |{ |
| | | type: "string" 0x4741-NA (0)
0x4740| 82 | . | size: 2 0x4741-0x4741.7 (1)
@ -3189,13 +3189,13 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x4750| 82 | . | size: 2 0x475c-0x475c.7 (1)
0x4750| 36 35 | 65 | value: "65" 0x475d-0x475e.7 (2)
| | | [2]{}: element 0x475f-0x4764.7 (6)
0x4750| 44| D| id: "tag_language" (0x447a) (Specifies the language of the tag specified, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the TagLanguageIETF Element is used within the same SimpleTag Element.) 0x475f-0x4760.7 (2)
0x4750| 44| D| id: "tag_language" (0x447a) (Specifies the language of the tag specified, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the TagLanguageBCP47 Element is used within the same SimpleTag Element.) 0x475f-0x4760.7 (2)
0x4760|7a |z |
| | | type: "string" 0x4761-NA (0)
0x4760| 83 | . | size: 3 0x4761-0x4761.7 (1)
0x4760| 65 6e 67 | eng | value: "eng" 0x4762-0x4764.7 (3)
| | | [3]{}: element 0x4765-0x4769.7 (5)
0x4760| 44 7b | D{ | id: "tag_language_ietf" (0x447b) (Specifies the language used in the TagString according to [@!BCP47] and using the IANA Language Subtag Registry [@!IANALangRegistry]. If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.) 0x4765-0x4766.7 (2)
0x4760| 44 7b | D{ | id: "tag_language_bcp47" (0x447b) (The language used in the TagString, in the [@!BCP47] form; see (#language-codes) on language codes. If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.) 0x4765-0x4766.7 (2)
| | | type: "string" 0x4767-NA (0)
0x4760| 82 | . | size: 2 0x4767-0x4767.7 (1)
0x4760| 65 6e | en | value: "en" 0x4768-0x4769.7 (2)
@ -3216,12 +3216,12 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x4780| 85 | . | size: 5 0x4781-0x4781.7 (1)
0x4780| 31 32 33 30 31 | 12301 | value: "12301" 0x4782-0x4786.7 (5)
| | | [2]{}: element 0x4787-0x478c.7 (6)
0x4780| 44 7a | Dz | id: "tag_language" (0x447a) (Specifies the language of the tag specified, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the TagLanguageIETF Element is used within the same SimpleTag Element.) 0x4787-0x4788.7 (2)
0x4780| 44 7a | Dz | id: "tag_language" (0x447a) (Specifies the language of the tag specified, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the TagLanguageBCP47 Element is used within the same SimpleTag Element.) 0x4787-0x4788.7 (2)
| | | type: "string" 0x4789-NA (0)
0x4780| 83 | . | size: 3 0x4789-0x4789.7 (1)
0x4780| 65 6e 67 | eng | value: "eng" 0x478a-0x478c.7 (3)
| | | [3]{}: element 0x478d-0x4791.7 (5)
0x4780| 44 7b | D{ | id: "tag_language_ietf" (0x447b) (Specifies the language used in the TagString according to [@!BCP47] and using the IANA Language Subtag Registry [@!IANALangRegistry]. If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.) 0x478d-0x478e.7 (2)
0x4780| 44 7b | D{ | id: "tag_language_bcp47" (0x447b) (The language used in the TagString, in the [@!BCP47] form; see (#language-codes) on language codes. If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.) 0x478d-0x478e.7 (2)
| | | type: "string" 0x478f-NA (0)
0x4780| 82| .| size: 2 0x478f-0x478f.7 (1)
0x4790|65 6e |en | value: "en" 0x4790-0x4791.7 (2)
@ -3245,12 +3245,12 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x47c0|2e 30 20 28 27 53 65 63 72 65 74 20 46 6f 72 20|.0 ('Secret For |
0x47d0|54 68 65 20 4d 61 64 27 29 20 36 34 2d 62 69 74|The Mad') 64-bit|
| | | [2]{}: element 0x47e0-0x47e5.7 (6)
0x47e0|44 7a |Dz | id: "tag_language" (0x447a) (Specifies the language of the tag specified, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the TagLanguageIETF Element is used within the same SimpleTag Element.) 0x47e0-0x47e1.7 (2)
0x47e0|44 7a |Dz | id: "tag_language" (0x447a) (Specifies the language of the tag specified, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the TagLanguageBCP47 Element is used within the same SimpleTag Element.) 0x47e0-0x47e1.7 (2)
| | | type: "string" 0x47e2-NA (0)
0x47e0| 83 | . | size: 3 0x47e2-0x47e2.7 (1)
0x47e0| 65 6e 67 | eng | value: "eng" 0x47e3-0x47e5.7 (3)
| | | [3]{}: element 0x47e6-0x47ea.7 (5)
0x47e0| 44 7b | D{ | id: "tag_language_ietf" (0x447b) (Specifies the language used in the TagString according to [@!BCP47] and using the IANA Language Subtag Registry [@!IANALangRegistry]. If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.) 0x47e6-0x47e7.7 (2)
0x47e0| 44 7b | D{ | id: "tag_language_bcp47" (0x447b) (The language used in the TagString, in the [@!BCP47] form; see (#language-codes) on language codes. If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.) 0x47e6-0x47e7.7 (2)
| | | type: "string" 0x47e8-NA (0)
0x47e0| 82 | . | size: 2 0x47e8-0x47e8.7 (1)
0x47e0| 65 6e | en | value: "en" 0x47e9-0x47ea.7 (2)
@ -3272,12 +3272,12 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x4810|32 30 32 32 2d 30 38 2d 32 38 20 31 38 3a 33 32|2022-08-28 18:32| value: "2022-08-28 18:32:21" 0x4810-0x4822.7 (19)
0x4820|3a 32 31 |:21 |
| | | [2]{}: element 0x4823-0x4828.7 (6)
0x4820| 44 7a | Dz | id: "tag_language" (0x447a) (Specifies the language of the tag specified, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the TagLanguageIETF Element is used within the same SimpleTag Element.) 0x4823-0x4824.7 (2)
0x4820| 44 7a | Dz | id: "tag_language" (0x447a) (Specifies the language of the tag specified, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the TagLanguageBCP47 Element is used within the same SimpleTag Element.) 0x4823-0x4824.7 (2)
| | | type: "string" 0x4825-NA (0)
0x4820| 83 | . | size: 3 0x4825-0x4825.7 (1)
0x4820| 65 6e 67 | eng | value: "eng" 0x4826-0x4828.7 (3)
| | | [3]{}: element 0x4829-0x482d.7 (5)
0x4820| 44 7b | D{ | id: "tag_language_ietf" (0x447b) (Specifies the language used in the TagString according to [@!BCP47] and using the IANA Language Subtag Registry [@!IANALangRegistry]. If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.) 0x4829-0x482a.7 (2)
0x4820| 44 7b | D{ | id: "tag_language_bcp47" (0x447b) (The language used in the TagString, in the [@!BCP47] form; see (#language-codes) on language codes. If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.) 0x4829-0x482a.7 (2)
| | | type: "string" 0x482b-NA (0)
0x4820| 82 | . | size: 2 0x482b-0x482b.7 (1)
0x4820| 65 6e | en | value: "en" 0x482c-0x482d.7 (2)
@ -3300,12 +3300,12 @@ $ fq dv sweep-with-DC.mkvmerge13.mka
0x4850|49 4f 4e 20 4e 55 4d 42 45 52 5f 4f 46 5f 46 52|ION NUMBER_OF_FR|
* |until 0x4873.7 (45) | |
| | | [2]{}: element 0x4874-0x4879.7 (6)
0x4870| 44 7a | Dz | id: "tag_language" (0x447a) (Specifies the language of the tag specified, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the TagLanguageIETF Element is used within the same SimpleTag Element.) 0x4874-0x4875.7 (2)
0x4870| 44 7a | Dz | id: "tag_language" (0x447a) (Specifies the language of the tag specified, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the TagLanguageBCP47 Element is used within the same SimpleTag Element.) 0x4874-0x4875.7 (2)
| | | type: "string" 0x4876-NA (0)
0x4870| 83 | . | size: 3 0x4876-0x4876.7 (1)
0x4870| 65 6e 67 | eng | value: "eng" 0x4877-0x4879.7 (3)
| | | [3]{}: element 0x487a-0x487e.7 (5)
0x4870| 44 7b | D{ | id: "tag_language_ietf" (0x447b) (Specifies the language used in the TagString according to [@!BCP47] and using the IANA Language Subtag Registry [@!IANALangRegistry]. If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.) 0x487a-0x487b.7 (2)
0x4870| 44 7b | D{ | id: "tag_language_bcp47" (0x447b) (The language used in the TagString, in the [@!BCP47] form; see (#language-codes) on language codes. If this Element is used, then any TagLanguage Elements used in the same SimpleTag **MUST** be ignored.) 0x487a-0x487b.7 (2)
| | | type: "string" 0x487c-NA (0)
0x4870| 82 | . | size: 2 0x487c-0x487c.7 (1)
0x4870| 65 6e| | en|| value: "en" 0x487d-0x487e.7 (2)

28
format/matroska/testdata/vorbis.fqtest vendored
View File

@ -49,7 +49,7 @@ $ fq -d matroska dv vorbis.mkv
0x0030|00 00 10 c7 |.... |
| | | elements[0:7]: 0x34-0x10fa.7 (4295)
| | | [0]{}: element 0x34-0x78.7 (69)
0x0030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains the Segment Position of other Top-Level Elements.) 0x34-0x37.7 (4)
0x0030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains seeking information of Top-Level Elements; see (#data-layout).) 0x34-0x37.7 (4)
| | | type: "master" 0x38-NA (0)
0x0030| c0 | . | size: 64 0x38-0x38.7 (1)
| | | elements[0:5]: 0x39-0x78.7 (64)
@ -65,12 +65,12 @@ $ fq -d matroska dv vorbis.mkv
0x0040| 8b | . | size: 11 0x41-0x41.7 (1)
| | | elements[0:2]: 0x42-0x4c.7 (11)
| | | [0]{}: element 0x42-0x48.7 (7)
0x0040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x42-0x43.7 (2)
0x0040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x42-0x43.7 (2)
| | | type: "binary" 0x44-NA (0)
0x0040| 84 | . | size: 4 0x44-0x44.7 (1)
0x0040| 15 49 a9 66 | .I.f | value: raw bits 0x45-0x48.7 (4)
| | | [1]{}: element 0x49-0x4c.7 (4)
0x0040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x49-0x4a.7 (2)
0x0040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x49-0x4a.7 (2)
| | | type: "uinteger" 0x4b-NA (0)
0x0040| 81 | . | size: 1 0x4b-0x4b.7 (1)
0x0040| a1 | . | value: 161 0x4c-0x4c.7 (1)
@ -80,12 +80,12 @@ $ fq -d matroska dv vorbis.mkv
0x0040| 8b| .| size: 11 0x4f-0x4f.7 (1)
| | | elements[0:2]: 0x50-0x5a.7 (11)
| | | [0]{}: element 0x50-0x56.7 (7)
0x0050|53 ab |S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x50-0x51.7 (2)
0x0050|53 ab |S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x50-0x51.7 (2)
| | | type: "binary" 0x52-NA (0)
0x0050| 84 | . | size: 4 0x52-0x52.7 (1)
0x0050| 16 54 ae 6b | .T.k | value: raw bits 0x53-0x56.7 (4)
| | | [1]{}: element 0x57-0x5a.7 (4)
0x0050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x57-0x58.7 (2)
0x0050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x57-0x58.7 (2)
| | | type: "uinteger" 0x59-NA (0)
0x0050| 81 | . | size: 1 0x59-0x59.7 (1)
0x0050| f1 | . | value: 241 0x5a-0x5a.7 (1)
@ -95,12 +95,12 @@ $ fq -d matroska dv vorbis.mkv
0x0050| 8c | . | size: 12 0x5d-0x5d.7 (1)
| | | elements[0:2]: 0x5e-0x69.7 (12)
| | | [0]{}: element 0x5e-0x64.7 (7)
0x0050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x5e-0x5f.7 (2)
0x0050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x5e-0x5f.7 (2)
| | | type: "binary" 0x60-NA (0)
0x0060|84 |. | size: 4 0x60-0x60.7 (1)
0x0060| 12 54 c3 67 | .T.g | value: raw bits 0x61-0x64.7 (4)
| | | [1]{}: element 0x65-0x69.7 (5)
0x0060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x65-0x66.7 (2)
0x0060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x65-0x66.7 (2)
| | | type: "uinteger" 0x67-NA (0)
0x0060| 82 | . | size: 2 0x67-0x67.7 (1)
0x0060| 0e 22 | ." | value: 3618 0x68-0x69.7 (2)
@ -110,12 +110,12 @@ $ fq -d matroska dv vorbis.mkv
0x0060| 8c | . | size: 12 0x6c-0x6c.7 (1)
| | | elements[0:2]: 0x6d-0x78.7 (12)
| | | [0]{}: element 0x6d-0x73.7 (7)
0x0060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x6d-0x6e.7 (2)
0x0060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x6d-0x6e.7 (2)
| | | type: "binary" 0x6f-NA (0)
0x0060| 84| .| size: 4 0x6f-0x6f.7 (1)
0x0070|1c 53 bb 6b |.S.k | value: raw bits 0x70-0x73.7 (4)
| | | [1]{}: element 0x74-0x78.7 (5)
0x0070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x74-0x75.7 (2)
0x0070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x74-0x75.7 (2)
| | | type: "uinteger" 0x76-NA (0)
0x0070| 82 | . | size: 2 0x76-0x76.7 (1)
0x0070| 10 ab | .. | value: 4267 0x77-0x78.7 (2)
@ -155,7 +155,7 @@ $ fq -d matroska dv vorbis.mkv
0x00f0| 4c 61 76 66 35 38| Lavf58| value: "Lavf58.45.100" 0xfa-0x106.7 (13)
0x0100|2e 34 35 2e 31 30 30 |.45.100 |
| | | [4]{}: element 0x107-0x119.7 (19)
0x0100| 73 a4 | s. | id: "segment_uid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits).) 0x107-0x108.7 (2)
0x0100| 73 a4 | s. | id: "segment_uuid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits). It is effectively a Universally Unique IDentifier stored in binary form [@!RFC4122].) 0x107-0x108.7 (2)
| | | type: "binary" 0x109-NA (0)
0x0100| 90 | . | size: 16 0x109-0x109.7 (1)
0x0100| 55 86 bb 9d 18 21| U....!| value: raw bits 0x10a-0x119.7 (16)
@ -198,13 +198,13 @@ $ fq -d matroska dv vorbis.mkv
0x0140| 81 | . | size: 1 0x149-0x149.7 (1)
0x0140| 00 | . | value: 0 0x14a-0x14a.7 (1)
| | | [3]{}: element 0x14b-0x151.7 (7)
0x0140| 22 b5 9c | ".. | id: "language" (0x22b59c) (Specifies the language of the track in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageIETF Element is used in the same TrackEntry.) 0x14b-0x14d.7 (3)
0x0140| 22 b5 9c | ".. | id: "language" (0x22b59c) (The language of the track, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageBCP47 Element is used in the same TrackEntry.) 0x14b-0x14d.7 (3)
| | | type: "string" 0x14e-NA (0)
0x0140| 83 | . | size: 3 0x14e-0x14e.7 (1)
0x0140| 75| u| value: "und" 0x14f-0x151.7 (3)
0x0150|6e 64 |nd |
| | | [4]{}: element 0x152-0x15b.7 (10)
0x0150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@!MatroskaCodec] for more info.) 0x152-0x152.7 (1)
0x0150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@?MatroskaCodec] for more info.) 0x152-0x152.7 (1)
| | | type: "string" 0x153-NA (0)
0x0150| 88 | . | size: 8 0x153-0x153.7 (1)
0x0150| 41 5f 56 4f 52 42 49 53 | A_VORBIS | value: "A_VORBIS" 0x154-0x15b.7 (8)
@ -279,7 +279,7 @@ $ fq -d matroska dv vorbis.mkv
0x01c0|86 81 d0 90 95 00 00 02 00 00 60 84 22 0c 31 20|..........`.".1 |
* |until 0xe55.7 (3231) | |
| | | [4]{}: element 0xe56-0xefa.7 (165)
0x0e50| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@!MatroskaTags].) 0xe56-0xe59.7 (4)
0x0e50| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@?MatroskaTags].) 0xe56-0xe59.7 (4)
| | | type: "master" 0xe5a-NA (0)
0x0e50| 40 9f | @. | size: 159 0xe5a-0xe5b.7 (2)
| | | elements[0:3]: 0xe5c-0xefa.7 (159)
@ -476,7 +476,7 @@ $ fq -d matroska dv vorbis.mkv
0x10f0| 81 | . | size: 1 0x10f2-0x10f2.7 (1)
0x10f0| 01 | . | value: 1 0x10f3-0x10f3.7 (1)
| | | [1]{}: element 0x10f4-0x10f7.7 (4)
0x10f0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position of the Cluster containing the associated Block.) 0x10f4-0x10f4.7 (1)
0x10f0| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position ((#segment-position)) of the Cluster containing the associated Block.) 0x10f4-0x10f4.7 (1)
| | | type: "uinteger" 0x10f5-NA (0)
0x10f0| 82 | . | size: 2 0x10f5-0x10f5.7 (1)
0x10f0| 0e c7 | .. | value: 3783 0x10f6-0x10f7.7 (2)

28
format/matroska/testdata/vp8.fqtest vendored
View File

@ -49,7 +49,7 @@ $ fq -d matroska dv vp8.mkv
0x0030|00 00 14 58 |...X |
| | | elements[0:7]: 0x34-0x148b.7 (5208)
| | | [0]{}: element 0x34-0x78.7 (69)
0x0030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains the Segment Position of other Top-Level Elements.) 0x34-0x37.7 (4)
0x0030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains seeking information of Top-Level Elements; see (#data-layout).) 0x34-0x37.7 (4)
| | | type: "master" 0x38-NA (0)
0x0030| c0 | . | size: 64 0x38-0x38.7 (1)
| | | elements[0:5]: 0x39-0x78.7 (64)
@ -65,12 +65,12 @@ $ fq -d matroska dv vp8.mkv
0x0040| 8b | . | size: 11 0x41-0x41.7 (1)
| | | elements[0:2]: 0x42-0x4c.7 (11)
| | | [0]{}: element 0x42-0x48.7 (7)
0x0040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x42-0x43.7 (2)
0x0040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x42-0x43.7 (2)
| | | type: "binary" 0x44-NA (0)
0x0040| 84 | . | size: 4 0x44-0x44.7 (1)
0x0040| 15 49 a9 66 | .I.f | value: raw bits 0x45-0x48.7 (4)
| | | [1]{}: element 0x49-0x4c.7 (4)
0x0040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x49-0x4a.7 (2)
0x0040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x49-0x4a.7 (2)
| | | type: "uinteger" 0x4b-NA (0)
0x0040| 81 | . | size: 1 0x4b-0x4b.7 (1)
0x0040| a1 | . | value: 161 0x4c-0x4c.7 (1)
@ -80,12 +80,12 @@ $ fq -d matroska dv vp8.mkv
0x0040| 8b| .| size: 11 0x4f-0x4f.7 (1)
| | | elements[0:2]: 0x50-0x5a.7 (11)
| | | [0]{}: element 0x50-0x56.7 (7)
0x0050|53 ab |S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x50-0x51.7 (2)
0x0050|53 ab |S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x50-0x51.7 (2)
| | | type: "binary" 0x52-NA (0)
0x0050| 84 | . | size: 4 0x52-0x52.7 (1)
0x0050| 16 54 ae 6b | .T.k | value: raw bits 0x53-0x56.7 (4)
| | | [1]{}: element 0x57-0x5a.7 (4)
0x0050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x57-0x58.7 (2)
0x0050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x57-0x58.7 (2)
| | | type: "uinteger" 0x59-NA (0)
0x0050| 81 | . | size: 1 0x59-0x59.7 (1)
0x0050| f1 | . | value: 241 0x5a-0x5a.7 (1)
@ -95,12 +95,12 @@ $ fq -d matroska dv vp8.mkv
0x0050| 8c | . | size: 12 0x5d-0x5d.7 (1)
| | | elements[0:2]: 0x5e-0x69.7 (12)
| | | [0]{}: element 0x5e-0x64.7 (7)
0x0050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x5e-0x5f.7 (2)
0x0050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x5e-0x5f.7 (2)
| | | type: "binary" 0x60-NA (0)
0x0060|84 |. | size: 4 0x60-0x60.7 (1)
0x0060| 12 54 c3 67 | .T.g | value: raw bits 0x61-0x64.7 (4)
| | | [1]{}: element 0x65-0x69.7 (5)
0x0060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x65-0x66.7 (2)
0x0060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x65-0x66.7 (2)
| | | type: "uinteger" 0x67-NA (0)
0x0060| 82 | . | size: 2 0x67-0x67.7 (1)
0x0060| 01 3f | .? | value: 319 0x68-0x69.7 (2)
@ -110,12 +110,12 @@ $ fq -d matroska dv vp8.mkv
0x0060| 8c | . | size: 12 0x6c-0x6c.7 (1)
| | | elements[0:2]: 0x6d-0x78.7 (12)
| | | [0]{}: element 0x6d-0x73.7 (7)
0x0060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x6d-0x6e.7 (2)
0x0060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x6d-0x6e.7 (2)
| | | type: "binary" 0x6f-NA (0)
0x0060| 84| .| size: 4 0x6f-0x6f.7 (1)
0x0070|1c 53 bb 6b |.S.k | value: raw bits 0x70-0x73.7 (4)
| | | [1]{}: element 0x74-0x78.7 (5)
0x0070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x74-0x75.7 (2)
0x0070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x74-0x75.7 (2)
| | | type: "uinteger" 0x76-NA (0)
0x0070| 82 | . | size: 2 0x76-0x76.7 (1)
0x0070| 14 3c | .< | value: 5180 0x77-0x78.7 (2)
@ -155,7 +155,7 @@ $ fq -d matroska dv vp8.mkv
0x00f0| 4c 61 76 66 35 38| Lavf58| value: "Lavf58.45.100" 0xfa-0x106.7 (13)
0x0100|2e 34 35 2e 31 30 30 |.45.100 |
| | | [4]{}: element 0x107-0x119.7 (19)
0x0100| 73 a4 | s. | id: "segment_uid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits).) 0x107-0x108.7 (2)
0x0100| 73 a4 | s. | id: "segment_uuid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits). It is effectively a Universally Unique IDentifier stored in binary form [@!RFC4122].) 0x107-0x108.7 (2)
| | | type: "binary" 0x109-NA (0)
0x0100| 90 | . | size: 16 0x109-0x109.7 (1)
0x0100| b6 f7 5d 8a 6f c3| ..].o.| value: raw bits 0x10a-0x119.7 (16)
@ -198,13 +198,13 @@ $ fq -d matroska dv vp8.mkv
0x0140| 81 | . | size: 1 0x148-0x148.7 (1)
0x0140| 00 | . | value: 0 0x149-0x149.7 (1)
| | | [3]{}: element 0x14a-0x150.7 (7)
0x0140| 22 b5 9c | ".. | id: "language" (0x22b59c) (Specifies the language of the track in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageIETF Element is used in the same TrackEntry.) 0x14a-0x14c.7 (3)
0x0140| 22 b5 9c | ".. | id: "language" (0x22b59c) (The language of the track, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageBCP47 Element is used in the same TrackEntry.) 0x14a-0x14c.7 (3)
| | | type: "string" 0x14d-NA (0)
0x0140| 83 | . | size: 3 0x14d-0x14d.7 (1)
0x0140| 75 6e| un| value: "und" 0x14e-0x150.7 (3)
0x0150|64 |d |
| | | [4]{}: element 0x151-0x157.7 (7)
0x0150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@!MatroskaCodec] for more info.) 0x151-0x151.7 (1)
0x0150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@?MatroskaCodec] for more info.) 0x151-0x151.7 (1)
| | | type: "string" 0x152-NA (0)
0x0150| 85 | . | size: 5 0x152-0x152.7 (1)
0x0150| 56 5f 56 50 38 | V_VP8 | value: "V_VP8" 0x153-0x157.7 (5)
@ -235,7 +235,7 @@ $ fq -d matroska dv vp8.mkv
0x0170| 81 | . | size: 1 0x171-0x171.7 (1)
0x0170| f0 | . | value: 240 0x172-0x172.7 (1)
| | | [4]{}: element 0x173-0x217.7 (165)
0x0170| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@!MatroskaTags].) 0x173-0x176.7 (4)
0x0170| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@?MatroskaTags].) 0x173-0x176.7 (4)
| | | type: "master" 0x177-NA (0)
0x0170| 40 9f | @. | size: 159 0x177-0x178.7 (2)
| | | elements[0:3]: 0x179-0x217.7 (159)
@ -402,7 +402,7 @@ $ fq -d matroska dv vp8.mkv
0x1480| 81 | . | size: 1 0x1483-0x1483.7 (1)
0x1480| 01 | . | value: 1 0x1484-0x1484.7 (1)
| | | [1]{}: element 0x1485-0x1488.7 (4)
0x1480| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position of the Cluster containing the associated Block.) 0x1485-0x1485.7 (1)
0x1480| f1 | . | id: "cue_cluster_position" (0xf1) (The Segment Position ((#segment-position)) of the Cluster containing the associated Block.) 0x1485-0x1485.7 (1)
| | | type: "uinteger" 0x1486-NA (0)
0x1480| 82 | . | size: 2 0x1486-0x1486.7 (1)
0x1480| 01 e4 | .. | value: 484 0x1487-0x1488.7 (2)

28
format/matroska/testdata/vp9.fqtest vendored
View File

@ -49,7 +49,7 @@ $ fq -d matroska dv vp9.mkv
0x0030|00 00 17 52 |...R |
| | | elements[0:7]: 0x34-0x1785.7 (5970)
| | | [0]{}: element 0x34-0x78.7 (69)
0x0030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains the Segment Position of other Top-Level Elements.) 0x34-0x37.7 (4)
0x0030| 11 4d 9b 74 | .M.t | id: "seek_head" (0x114d9b74) (Contains seeking information of Top-Level Elements; see (#data-layout).) 0x34-0x37.7 (4)
| | | type: "master" 0x38-NA (0)
0x0030| c0 | . | size: 64 0x38-0x38.7 (1)
| | | elements[0:5]: 0x39-0x78.7 (64)
@ -65,12 +65,12 @@ $ fq -d matroska dv vp9.mkv
0x0040| 8b | . | size: 11 0x41-0x41.7 (1)
| | | elements[0:2]: 0x42-0x4c.7 (11)
| | | [0]{}: element 0x42-0x48.7 (7)
0x0040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x42-0x43.7 (2)
0x0040| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x42-0x43.7 (2)
| | | type: "binary" 0x44-NA (0)
0x0040| 84 | . | size: 4 0x44-0x44.7 (1)
0x0040| 15 49 a9 66 | .I.f | value: raw bits 0x45-0x48.7 (4)
| | | [1]{}: element 0x49-0x4c.7 (4)
0x0040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x49-0x4a.7 (2)
0x0040| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x49-0x4a.7 (2)
| | | type: "uinteger" 0x4b-NA (0)
0x0040| 81 | . | size: 1 0x4b-0x4b.7 (1)
0x0040| a1 | . | value: 161 0x4c-0x4c.7 (1)
@ -80,12 +80,12 @@ $ fq -d matroska dv vp9.mkv
0x0040| 8b| .| size: 11 0x4f-0x4f.7 (1)
| | | elements[0:2]: 0x50-0x5a.7 (11)
| | | [0]{}: element 0x50-0x56.7 (7)
0x0050|53 ab |S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x50-0x51.7 (2)
0x0050|53 ab |S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x50-0x51.7 (2)
| | | type: "binary" 0x52-NA (0)
0x0050| 84 | . | size: 4 0x52-0x52.7 (1)
0x0050| 16 54 ae 6b | .T.k | value: raw bits 0x53-0x56.7 (4)
| | | [1]{}: element 0x57-0x5a.7 (4)
0x0050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x57-0x58.7 (2)
0x0050| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x57-0x58.7 (2)
| | | type: "uinteger" 0x59-NA (0)
0x0050| 81 | . | size: 1 0x59-0x59.7 (1)
0x0050| f1 | . | value: 241 0x5a-0x5a.7 (1)
@ -95,12 +95,12 @@ $ fq -d matroska dv vp9.mkv
0x0050| 8c | . | size: 12 0x5d-0x5d.7 (1)
| | | elements[0:2]: 0x5e-0x69.7 (12)
| | | [0]{}: element 0x5e-0x64.7 (7)
0x0050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x5e-0x5f.7 (2)
0x0050| 53 ab| S.| id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x5e-0x5f.7 (2)
| | | type: "binary" 0x60-NA (0)
0x0060|84 |. | size: 4 0x60-0x60.7 (1)
0x0060| 12 54 c3 67 | .T.g | value: raw bits 0x61-0x64.7 (4)
| | | [1]{}: element 0x65-0x69.7 (5)
0x0060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x65-0x66.7 (2)
0x0060| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x65-0x66.7 (2)
| | | type: "uinteger" 0x67-NA (0)
0x0060| 82 | . | size: 2 0x67-0x67.7 (1)
0x0060| 01 3f | .? | value: 319 0x68-0x69.7 (2)
@ -110,12 +110,12 @@ $ fq -d matroska dv vp9.mkv
0x0060| 8c | . | size: 12 0x6c-0x6c.7 (1)
| | | elements[0:2]: 0x6d-0x78.7 (12)
| | | [0]{}: element 0x6d-0x73.7 (7)
0x0060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary ID corresponding to the Element name.) 0x6d-0x6e.7 (2)
0x0060| 53 ab | S. | id: "seek_id" (0x53ab) (The binary EBML ID of a Top-Level Element.) 0x6d-0x6e.7 (2)
| | | type: "binary" 0x6f-NA (0)
0x0060| 84| .| size: 4 0x6f-0x6f.7 (1)
0x0070|1c 53 bb 6b |.S.k | value: raw bits 0x70-0x73.7 (4)
| | | [1]{}: element 0x74-0x78.7 (5)
0x0070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position of the Element.) 0x74-0x75.7 (2)
0x0070| 53 ac | S. | id: "seek_position" (0x53ac) (The Segment Position ((#segment-position)) of a Top-Level Element.) 0x74-0x75.7 (2)
| | | type: "uinteger" 0x76-NA (0)
0x0070| 82 | . | size: 2 0x76-0x76.7 (1)
0x0070| 17 36 | .6 | value: 5942 0x77-0x78.7 (2)
@ -155,7 +155,7 @@ $ fq -d matroska dv vp9.mkv
0x00f0| 4c 61 76 66 35 38| Lavf58| value: "Lavf58.45.100" 0xfa-0x106.7 (13)
0x0100|2e 34 35 2e 31 30 30 |.45.100 |
| | | [4]{}: element 0x107-0x119.7 (19)
0x0100| 73 a4 | s. | id: "segment_uid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits).) 0x107-0x108.7 (2)
0x0100| 73 a4 | s. | id: "segment_uuid" (0x73a4) (A randomly generated unique ID to identify the Segment amongst many others (128 bits). It is effectively a Universally Unique IDentifier stored in binary form [@!RFC4122].) 0x107-0x108.7 (2)
| | | type: "binary" 0x109-NA (0)
0x0100| 90 | . | size: 16 0x109-0x109.7 (1)
0x0100| 79 44 d8 c6 65 1a| yD..e.| value: raw bits 0x10a-0x119.7 (16)
@ -198,13 +198,13 @@ $ fq -d matroska dv vp9.mkv
0x0140| 81 | . | size: 1 0x148-0x148.7 (1)
0x0140| 00 | . | value: 0 0x149-0x149.7 (1)
| | | [3]{}: element 0x14a-0x150.7 (7)
0x0140| 22 b5 9c | ".. | id: "language" (0x22b59c) (Specifies the language of the track in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageIETF Element is used in the same TrackEntry.) 0x14a-0x14c.7 (3)
0x0140| 22 b5 9c | ".. | id: "language" (0x22b59c) (The language of the track, in the Matroska languages form; see (#language-codes) on language codes. This Element **MUST** be ignored if the LanguageBCP47 Element is used in the same TrackEntry.) 0x14a-0x14c.7 (3)
| | | type: "string" 0x14d-NA (0)
0x0140| 83 | . | size: 3 0x14d-0x14d.7 (1)
0x0140| 75 6e| un| value: "und" 0x14e-0x150.7 (3)
0x0150|64 |d |
| | | [4]{}: element 0x151-0x157.7 (7)
0x0150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@!MatroskaCodec] for more info.) 0x151-0x151.7 (1)
0x0150| 86 | . | id: "codec_id" (0x86) (An ID corresponding to the codec, see [@?MatroskaCodec] for more info.) 0x151-0x151.7 (1)
| | | type: "string" 0x152-NA (0)
0x0150| 85 | . | size: 5 0x152-0x152.7 (1)
0x0150| 56 5f 56 50 39 | V_VP9 | value: "V_VP9" 0x153-0x157.7 (5)
@ -235,7 +235,7 @@ $ fq -d matroska dv vp9.mkv
0x0170| 81 | . | size: 1 0x171-0x171.7 (1)
0x0170| f0 | . | value: 240 0x172-0x172.7 (1)
| | | [4]{}: element 0x173-0x21b.7 (169)
0x0170| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@!MatroskaTags].) 0x173-0x176.7 (4)
0x0170| 12 54 c3 67 | .T.g | id: "tags" (0x1254c367) (Element containing metadata describing Tracks, Editions, Chapters, Attachments, or the Segment as a whole. A list of valid tags can be found in [@?MatroskaTags].) 0x173-0x176.7 (4)
| | | type: "master" 0x177-NA (0)
0x0170| 40 a3 | @. | size: 163 0x177-0x178.7 (2)
| | | elements[0:3]: 0x179-0x21b.7 (163)
@ -403,7 +403,7 @@ $ fq -d matroska dv vp9.mkv
0x1770| 81 | . | size: 1 0x177d-0x177d.7 (1)
0x1770| 01 | . | value: 1 0x177e-0x177e.7 (1)
| | | [1]{}: element 0x177f-0x1782.7 (4)
0x1770| f1| .| id: "cue_cluster_position" (0xf1) (The Segment Position of the Cluster containing the associated Block.) 0x177f-0x177f.7 (1)
0x1770| f1| .| id: "cue_cluster_position" (0xf1) (The Segment Position ((#segment-position)) of the Cluster containing the associated Block.) 0x177f-0x177f.7 (1)
| | | type: "uinteger" 0x1780-NA (0)
0x1780|82 |. | size: 2 0x1780-0x1780.7 (1)
0x1780| 01 e8 | .. | value: 488 0x1781-0x1782.7 (2)

21
format/mp3/testdata/help_mp3.fqtest vendored Normal file
View File

@ -0,0 +1,21 @@
$ fq -h mp3
mp3: MP3 file decoder
Options
=======
max_sync_seek=32768 Max byte distance to next sync
max_unique_header_configs=5 Max number of unique frame header configs allowed
Decode examples
===============
# Decode file as mp3
$ fq -d mp3 . file
# Decode value as mp3
... | mp3
# Decode file using mp3 options
$ fq -d mp3 -o max_sync_seek=32768 -o max_unique_header_configs=5 . file
# Decode value as mp3
... | mp3({max_sync_seek:32768,max_unique_header_configs:5})

117
format/mp4/boxes.go
View File

@ -62,21 +62,118 @@ var subTypeNames = scalar.StrToDescription{
}
var dataFormatNames = scalar.StrToDescription{
// additional codecs
"apch": "Apple ProRes 422 High Quality",
"apcn": "Apple ProRes 422 Standard Definition",
"apcs": "Apple ProRes 422 LT",
"apco": "Apple ProRes 422 Proxy",
"ap4h": "Apple ProRes 4444",
"fLaC": "Fres Lossless Audio Codec",
"Opus": "Xiph Opus",
"vp09": "VP9",
"avc1": "Advanced Video Coding / H.264 / MPEG-4 Part 10",
"hev1": "High Efficiency Video Coding / H.265 / MPEG-H Part 2",
"hvc1": "High Efficiency Video Coding / H.265 / MPEG-H Part 2",
"av01": "AV1",
"mp4a": "MPEG Audio",
"mp4v": "MPEG Video",
"jpeg": "JPEG Image",
// codecs from https://mp4ra.org/
"3gvo": "3GPP Video Orientation",
"a3d1": "Multiview Video Coding",
"a3d2": "Multiview Video Coding",
"a3d3": "Multiview Video Coding",
"a3d4": "Multiview Video Coding",
"a3ds": "Auro-Cx 3D audio",
"ac-3": "AC-3 audio",
"ac-4": "AC-4 audio",
"alac": "Apple lossless audio codec",
"alaw": "a-Law",
"av01": "AV1 video",
"avc1": "Advanced Video Coding",
"avc2": "Advanced Video Coding",
"avc3": "Advanced Video Coding",
"avc4": "Advanced Video Coding",
"avcp": "Advanced Video Coding Parameters",
"dra1": "DRA Audio",
"drac": "Dirac Video Coder",
"dts-": "Dependent base layer for DTS layered audio",
"dts+": "Enhancement layer for DTS layered audio",
"dtsc": "DTS Coherent Acoustics audio",
"dtse": "DTS Express low bit rate audio, also known as DTS LBR",
"dtsh": "DTS-HD High Resolution Audio",
"dtsl": "DTS-HD Master Audio",
"dtsx": "DTS:X",
"dvav": "AVC-based “Dolby Vision”",
"dvhe": "HEVC-based “Dolby Vision”",
"ec-3": "Enhanced AC-3 audio",
"enca": "Encrypted/Protected audio",
"encf": "Encrypted/Protected font",
"encm": "Encrypted/Protected metadata stream",
"encs": "Encrypted Systems stream",
"enct": "Encrypted Text",
"encv": "Encrypted/protected video",
"fdp ": "File delivery hints",
"fLaC": "Fres Lossless Audio Codec",
"g719": "ITU-T Recommendation G.719 (2008)",
"g726": "ITU-T Recommendation G.726 (1990)",
"hev1": "High Efficiency Video Coding",
"hvc1": "High Efficiency Video Coding",
"hvt1": "High Efficiency Video Coding",
"ixse": "DVB Track Level Index Track",
"lhe1": "Layered High Efficiency Video Coding",
"lht1": "Layered High Efficiency Video Coding",
"lhv1": "Layered High Efficiency Video Coding",
"m2ts": "MPEG-2 transport stream for DMB",
"m4ae": "MPEG-4 Audio Enhancement MP4v1/2",
"mett": "Text timed metadata that is not XML",
"metx": "XML timed metadata",
"mha1": "MPEG-H Audio (single stream, uncapsulated)",
"mha2": "MPEG-H Audio (multi-stream, unencapsulated)",
"mhm1": "MPEG-H Audio (single stream, MHAS encapsulated)",
"mhm2": "MPEG-H Audio (multi-stream, MHAS encapsulated)",
"mjp2": "Motion JPEG 2000",
"mlix": "DVB Movie level index track",
"mlpa": "MLP Audio",
"mp4a": "MPEG-4 Audio",
"mp4s": "MPEG-4 Systems",
"mp4v": "MPEG-4 Visual",
"mvc1": "Multiview coding",
"mvc2": "Multiview coding",
"mvc3": "Multiview coding",
"mvc4": "Multiview coding",
"mvd1": "Multiview coding",
"mvd2": "Multiview coding",
"mvd3": "Multiview coding",
"mvd4": "Multiview coding",
"oksd": "OMA Keys",
"Opus": "Opus audio coding",
"pm2t": "Protected MPEG-2 Transport",
"prtp": "Protected RTP Reception",
"raw ": "Uncompressed audio",
"resv": "Restricted Video",
"rm2t": "MPEG-2 Transport Reception",
"rrtp": "RTP reception",
"rsrp": "SRTP Reception",
"rtmd": "Real Time Metadata Sample Entry(XAVC Format)",
"rtp ": "RTP Hints",
"s263": "ITU H.263 video (3GPP format)",
"samr": "Narrowband AMR voice",
"sawb": "Wideband AMR voice",
"sawp": "Extended AMR-WB (AMR-WB+)",
"sevc": "EVRC Voice",
"sm2t": "MPEG-2 Transport Server",
"sqcp": "13K Voice",
"srtp": "SRTP Hints",
"ssmv": "SMV Voice",
"STGS": "Subtitle Sample Entry (HMMP)",
"stpp": "Subtitles (Timed Text)",
"svc1": "Scalable Video Coding",
"svc2": "Scalable Video Coding",
"svcM": "SVC Metadata",
"tc64": "64 bit timecode samples",
"tmcd": "32 bit timecode samples",
"twos": "Uncompressed 16-bit audio",
"tx3g": "Timed Text stream",
"ulaw": "Samples have been compressed using uLaw 2:1.",
"unid": "Dynamic Range Control (DRC) data",
"urim": "Binary timed metadata identified by URI",
"vc-1": "SMPTE VC-1",
"vp08": "VP8 video",
"vp09": "VP9 video",
"wvtt": "WebVTT",
}
var (
@ -113,7 +210,7 @@ func decodeLang(d *decode.D) string {
// Quicktime time seconds in January 1, 1904 UTC
var quicktimeEpochDate = time.Date(1904, time.January, 4, 0, 0, 0, 0, time.UTC)
var quicktimeEpoch = scalar.DescriptionActualUTime(quicktimeEpochDate, time.RFC3339)
var quicktimeEpoch = scalar.DescriptionTimeFn(scalar.S.TryActualU, quicktimeEpochDate, time.RFC3339)
func decodeFieldMatrix(d *decode.D, name string) {
d.FieldStruct(name, func(d *decode.D) {

6
format/mp4/mp4.go
View File

@ -26,6 +26,7 @@ import (
)
//go:embed mp4.jq
//go:embed mp4.md
var mp4FS embed.FS
var aacFrameFormat decode.Group
@ -55,7 +56,7 @@ var vpxCCRFormat decode.Group
func init() {
interp.RegisterFormat(decode.Format{
Name: format.MP4,
Description: "ISOBMFF MPEG-4 part 12 and similar",
Description: "ISOBMFF, QuickTime and similar",
Groups: []string{
format.PROBE,
format.IMAGE, // avif
@ -90,7 +91,6 @@ func init() {
{Names: []string{format.VP9_FRAME}, Group: &vp9FrameFormat},
{Names: []string{format.VPX_CCR}, Group: &vpxCCRFormat},
},
Functions: []string{"_help"},
})
interp.RegisterFS(mp4FS)
}
@ -271,7 +271,7 @@ func mp4Tracks(d *decode.D, ctx *decodeContext) {
}
}
d.FieldValueStr("data_foramt", trackSDDataFormat, dataFormatNames)
d.FieldValueStr("data_format", trackSDDataFormat, dataFormatNames)
switch trackSDDataFormat {
case "lpcm",

12
format/mp4/mp4.jq
View File

@ -12,15 +12,3 @@ def mp4_path:
| format_root
| mp4_path($c)
);
def _mp4__help:
{ notes: "Support `mp4_path`",
examples: [
{comment: "Lookup box decode value using `mp4_path`", expr: "mp4_path(\".moov.trak[1]\")"},
{comment: "Return `mp4_path` string for a box decode value", expr: "grep_by(.type == \"trak\") | mp4_path"}
],
links: [
{title: "ISO/IEC base media file format (MPEG-4 Part 12)", url: "https://en.wikipedia.org/wiki/ISO/IEC_base_media_file_format"},
{title: "Quicktime file format", url: "https://developer.apple.com/standards/qtff-2001.pdf"}
]
};

37
format/mp4/mp4.md Normal file
View File

@ -0,0 +1,37 @@
### Lookup mp4 box using a mp4 box path.
```sh
# <decode value box> | mp4_path($path) -> <decode value box>
$ fq 'mp4_path(".moov.trak[1]")' file.mp4
```
### Get mp4 box path for a decode value box.
```sh
# <decode value box> | mp4_path -> string
$ fq 'grep_by(.type == "trak") | mp4_path' file.mp4
```
### Force decode a single box
```sh
$ fq -n '"AAAAHGVsc3QAAAAAAAAAAQAAADIAAAQAAAEAAA==" | frombase64 | mp4({force:true}) | d'
```
### Speed up decoding by not decoding samples
```sh
# manually decode first sample as a aac_frame
$ fq -o decode_samples=false '.tracks[0].samples[0] | aac_frame | d' file.mp4
```
### Entries for first edit list as values
```sh
$ fq 'first(grep_by(.type=="elst").entries) | tovalue' file.mp4
```
### References
- [ISO/IEC base media file format (MPEG-4 Part 12)](https://en.wikipedia.org/wiki/ISO/IEC_base_media_file_format)
- [Quicktime file format](https://developer.apple.com/standards/qtff-2001.pdf)

4
format/mp4/testdata/aac.fqtest vendored
View File

@ -192,7 +192,7 @@ $ fq -d mp4 dv aac.mp4
| | | boxes[0:1]: 0x437-0x490.7 (90)
| | | [0]{}: box 0x437-0x490.7 (90)
0x430| 00 00 00 5a | ...Z | size: 90 0x437-0x43a.7 (4)
0x430| 6d 70 34 61 | mp4a | type: "mp4a" (MPEG Audio) 0x43b-0x43e.7 (4)
0x430| 6d 70 34 61 | mp4a | type: "mp4a" (MPEG-4 Audio) 0x43b-0x43e.7 (4)
0x430| 00| .| reserved: raw bits 0x43f-0x444.7 (6)
0x440|00 00 00 00 00 |..... |
0x440| 00 01 | .. | data_reference_index: 1 0x445-0x446.7 (2)
@ -429,4 +429,4 @@ $ fq -d mp4 dv aac.mp4
0x290|b4 |. | [1]: raw bits byte_align 0x290.6-0x290.7 (0.2)
0x290| 70 | p | [2]: raw bits data 0x291-0x291.7 (1)
| | | id: 1 0x59d-NA (0)
| | | data_foramt: "mp4a" (MPEG Audio) 0x59d-NA (0)
| | | data_format: "mp4a" (MPEG-4 Audio) 0x59d-NA (0)

4
format/mp4/testdata/av1.fqtest vendored
View File

@ -178,7 +178,7 @@ $ fq -d mp4 dv av1.mp4
| | | boxes[0:1]: 0x1369-0x13f3.7 (139)
| | | [0]{}: box 0x1369-0x13f3.7 (139)
0x1360| 00 00 00 8b | .... | size: 139 0x1369-0x136c.7 (4)
0x1360| 61 76 30| av0| type: "av01" (AV1) 0x136d-0x1370.7 (4)
0x1360| 61 76 30| av0| type: "av01" (AV1 video) 0x136d-0x1370.7 (4)
0x1370|31 |1 |
0x1370| 00 00 00 00 00 00 | ...... | reserved: raw bits 0x1371-0x1376.7 (6)
0x1370| 00 01 | .. | data_reference_index: 1 0x1377-0x1378.7 (2)
@ -341,4 +341,4 @@ $ fq -d mp4 dv av1.mp4
0x0050|f6 0a 4f ae f3 fe ec e7 30 4f 3f 13 9c 75 c9 6a|..O.....0O?..u.j| data: raw bits 0x50-0x11bf.7 (4464)
* |until 0x11bf.7 (4464) | |
| | | id: 1 0x14b2-NA (0)
| | | data_foramt: "av01" (AV1) 0x14b2-NA (0)
| | | data_format: "av01" (AV1 video) 0x14b2-NA (0)

4
format/mp4/testdata/avc.fqtest vendored
View File

@ -196,7 +196,7 @@ $ fq -d mp4 dv avc.mp4
| | | boxes[0:1]: 0xf2a-0xfd9.7 (176)
| | | [0]{}: box 0xf2a-0xfd9.7 (176)
0x00f20| 00 00 00 b0 | .... | size: 176 0xf2a-0xf2d.7 (4)
0x00f20| 61 76| av| type: "avc1" (Advanced Video Coding / H.264 / MPEG-4 Part 10) 0xf2e-0xf31.7 (4)
0x00f20| 61 76| av| type: "avc1" (Advanced Video Coding) 0xf2e-0xf31.7 (4)
0x00f30|63 31 |c1 |
0x00f30| 00 00 00 00 00 00 | ...... | reserved: raw bits 0xf32-0xf37.7 (6)
0x00f30| 00 01 | .. | data_reference_index: 1 0xf38-0xf39.7 (2)
@ -514,4 +514,4 @@ $ fq -d mp4 dv avc.mp4
0x00d50|79 0a ff 01 f9 2d 04 d3 29 fe 4d 76 42 26 f6 cd|y....-..).MvB&..|
* |until 0xd80.7 (51) | |
| | | id: 1 0x10e0-NA (0)
| | | data_foramt: "avc1" (Advanced Video Coding / H.264 / MPEG-4 Part 10) 0x10e0-NA (0)
| | | data_format: "avc1" (Advanced Video Coding) 0x10e0-NA (0)

12
format/mp4/testdata/dash.fqtest vendored
View File

@ -204,7 +204,7 @@ $ fq -d mp4 dv dash_audio_init.mp4
| | | boxes[0:1]: 0x218-0x265.7 (78)
| | | [0]{}: box 0x218-0x265.7 (78)
0x210| 00 00 00 4e | ...N | size: 78 0x218-0x21b.7 (4)
0x210| 6d 70 34 61| mp4a| type: "mp4a" (MPEG Audio) 0x21c-0x21f.7 (4)
0x210| 6d 70 34 61| mp4a| type: "mp4a" (MPEG-4 Audio) 0x21c-0x21f.7 (4)
0x220|00 00 00 00 00 00 |...... | reserved: raw bits 0x220-0x225.7 (6)
0x220| 00 01 | .. | data_reference_index: 1 0x226-0x227.7 (2)
0x220| 00 00 | .. | version: 0 0x228-0x229.7 (2)
@ -348,7 +348,7 @@ $ fq -d mp4 dv dash_audio_init.mp4
| | | tracks[0:1]: 0x330-NA (0)
| | | [0]{}: track 0x330-NA (0)
| | | id: 1 0x330-NA (0)
| | | data_foramt: "mp4a" (MPEG Audio) 0x330-NA (0)
| | | data_format: "mp4a" (MPEG-4 Audio) 0x330-NA (0)
| | | samples[0:0]: 0x330-NA (0)
$ fq -d mp4 dv dash_audio_1.m4s
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f|0123456789abcdef|.{}: dash_audio_1.m4s (mp4) 0x0-0x4eb.7 (1260)
@ -491,7 +491,7 @@ $ fq -d mp4 dv dash_audio_1.m4s
* |until 0x4e6.7 (193) | |
0x4e0| 01 18 81 b4 70| | ....p| | [5]: raw bits sample 0x4e7-0x4eb.7 (5)
| | | id: 1 0x4ec-NA (0)
| | | data_foramt: "unknown" 0x4ec-NA (0)
| | | data_format: "unknown" 0x4ec-NA (0)
$ fq -d mp4 dv dash_video_init.mp4
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f|0123456789abcdef|.{}: dash_video_init.mp4 (mp4) 0x0-0x332.7 (819)
| | | boxes[0:2]: 0x0-0x332.7 (819)
@ -696,7 +696,7 @@ $ fq -d mp4 dv dash_video_init.mp4
| | | boxes[0:1]: 0x21c-0x2a2.7 (135)
| | | [0]{}: box 0x21c-0x2a2.7 (135)
0x0210| 00 00 00 87| ....| size: 135 0x21c-0x21f.7 (4)
0x0220|61 76 63 31 |avc1 | type: "avc1" (Advanced Video Coding / H.264 / MPEG-4 Part 10) 0x220-0x223.7 (4)
0x0220|61 76 63 31 |avc1 | type: "avc1" (Advanced Video Coding) 0x220-0x223.7 (4)
0x0220| 00 00 00 00 00 00 | ...... | reserved: raw bits 0x224-0x229.7 (6)
0x0220| 00 01 | .. | data_reference_index: 1 0x22a-0x22b.7 (2)
0x0220| 00 00 | .. | version: 0 0x22c-0x22d.7 (2)
@ -886,7 +886,7 @@ $ fq -d mp4 dv dash_video_init.mp4
| | | tracks[0:1]: 0x333-NA (0)
| | | [0]{}: track 0x333-NA (0)
| | | id: 1 0x333-NA (0)
| | | data_foramt: "avc1" (Advanced Video Coding / H.264 / MPEG-4 Part 10) 0x333-NA (0)
| | | data_format: "avc1" (Advanced Video Coding) 0x333-NA (0)
| | | samples[0:0]: 0x333-NA (0)
$ fq -d mp4 dv dash_video_1.m4s
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f|0123456789abcdef|.{}: dash_video_1.m4s (mp4) 0x0-0x1fd0.7 (8145)
@ -1001,4 +1001,4 @@ $ fq -d mp4 dv dash_video_1.m4s
0x1720|53 23 af ff f2 50 06 7f 30 02 17 55 d4 5a 6f db|S#...P..0..U.Zo.|
* |until 0x1fd0.7 (end) (2240) | |
| | | id: 1 0x1fd1-NA (0)
| | | data_foramt: "unknown" 0x1fd1-NA (0)
| | | data_format: "unknown" 0x1fd1-NA (0)

2
format/mp4/testdata/decode_samples.fqtest vendored
View File

@ -14,4 +14,4 @@ $ fq -o decode_samples=false -d mp4 '.tracks | dv' aac.mp4
0x280| 01 18 81| ...| [3]: raw bits sample 0x28d-0x291.7 (5)
0x290|b4 70 |.p |
| | | id: 1 0x59d-NA (0)
| | | data_foramt: "mp4a" (MPEG Audio) 0x59d-NA (0)
| | | data_format: "mp4a" (MPEG-4 Audio) 0x59d-NA (0)

2
format/mp4/testdata/flac.fqtest vendored
View File

@ -372,4 +372,4 @@ $ fq -d mp4 dv flac.mp4
0x280| 00 | . | byte_align: 0 (valid) 0x287.2-0x287.7 (0.6)
0x280| 82 cb | .. | footer_crc: "82cb" (raw bits) (valid) 0x288-0x289.7 (2)
| | | id: 1 0x543-NA (0)
| | | data_foramt: "fLaC" (Fres Lossless Audio Codec) 0x543-NA (0)
| | | data_format: "fLaC" (Fres Lossless Audio Codec) 0x543-NA (0)

8
format/mp4/testdata/fragmented.fqtest vendored
View File

@ -154,7 +154,7 @@ $ fq -d mp4 dv fragmented.mp4
| | | boxes[0:1]: 0x1a9-0x23f.7 (151)
| | | [0]{}: box 0x1a9-0x23f.7 (151)
0x001a0| 00 00 00 97 | .... | size: 151 0x1a9-0x1ac.7 (4)
0x001a0| 61 76 63| avc| type: "avc1" (Advanced Video Coding / H.264 / MPEG-4 Part 10) 0x1ad-0x1b0.7 (4)
0x001a0| 61 76 63| avc| type: "avc1" (Advanced Video Coding) 0x1ad-0x1b0.7 (4)
0x001b0|31 |1 |
0x001b0| 00 00 00 00 00 00 | ...... | reserved: raw bits 0x1b1-0x1b6.7 (6)
0x001b0| 00 01 | .. | data_reference_index: 1 0x1b7-0x1b8.7 (2)
@ -411,7 +411,7 @@ $ fq -d mp4 dv fragmented.mp4
| | | boxes[0:1]: 0x391-0x3fe.7 (110)
| | | [0]{}: box 0x391-0x3fe.7 (110)
0x00390| 00 00 00 6e | ...n | size: 110 0x391-0x394.7 (4)
0x00390| 6d 70 34 61 | mp4a | type: "mp4a" (MPEG Audio) 0x395-0x398.7 (4)
0x00390| 6d 70 34 61 | mp4a | type: "mp4a" (MPEG-4 Audio) 0x395-0x398.7 (4)
0x00390| 00 00 00 00 00 00 | ...... | reserved: raw bits 0x399-0x39e.7 (6)
0x00390| 00| .| data_reference_index: 1 0x39f-0x3a0.7 (2)
0x003a0|01 |. |
@ -1167,7 +1167,7 @@ $ fq -d mp4 dv fragmented.mp4
0x02240|86 f8 14 d8 53 23 af ff f2 50 06 7f 30 02 17 55|....S#...P..0..U|
* |until 0x2af4.7 (2234) | |
| | | id: 1 0x2bb4-NA (0)
| | | data_foramt: "avc1" (Advanced Video Coding / H.264 / MPEG-4 Part 10) 0x2bb4-NA (0)
| | | data_format: "avc1" (Advanced Video Coding) 0x2bb4-NA (0)
| | | [1]{}: track 0x13e0-0x2bb3.7 (6100)
| | | samples[0:6]: 0x13e0-0x2af9.7 (5914)
|00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f|0123456789abcdef| [0][0:4]: sample (aac_frame) 0x13e0-0x14ad.7 (206)
@ -1270,4 +1270,4 @@ $ fq -d mp4 dv fragmented.mp4
0x02af0| b4 | . | [1]: raw bits byte_align 0x2af8.6-0x2af8.7 (0.2)
0x02af0| 70 | p | [2]: raw bits data 0x2af9-0x2af9.7 (1)
| | | id: 2 0x2bb4-NA (0)
| | | data_foramt: "mp4a" (MPEG Audio) 0x2bb4-NA (0)
| | | data_format: "mp4a" (MPEG-4 Audio) 0x2bb4-NA (0)

54
format/mp4/testdata/help_mp4.fqtest vendored Normal file
View File

@ -0,0 +1,54 @@
$ fq -h mp4
mp4: ISOBMFF, QuickTime and similar decoder
Options
=======
allow_truncated=false Allow box to be truncated
decode_samples=true Decode supported media samples
Decode examples
===============
# Decode file as mp4
$ fq -d mp4 . file
# Decode value as mp4
... | mp4
# Decode file using mp4 options
$ fq -d mp4 -o allow_truncated=false -o decode_samples=true . file
# Decode value as mp4
... | mp4({allow_truncated:false,decode_samples:true})
Lookup mp4 box using a mp4 box path.
====================================
# <decode value box> | mp4_path($path) -> <decode value box>
$ fq 'mp4_path(".moov.trak[1]")' file.mp4
Get mp4 box path for a decode value box.
========================================
# <decode value box> | mp4_path -> string
$ fq 'grep_by(.type == "trak") | mp4_path' file.mp4
Force decode a single box
=========================
$ fq -n '"AAAAHGVsc3QAAAAAAAAAAQAAADIAAAQAAAEAAA==" | frombase64 | mp4({force:true}) | d'
Speed up decoding by not decoding samples
=========================================
# manually decode first sample as a aac_frame
$ fq -o decode_samples=false '.tracks[0].samples[0] | aac_frame | d' file.mp4
Entries for first edit list as values
=====================================
$ fq 'first(grep_by(.type=="elst").entries) | tovalue' file.mp4
References
==========
- ISO/IEC base media file format (MPEG-4 Part 12) (https://en.wikipedia.org/wiki/ISO/IEC_base_media_file_format)
- Quicktime file format (https://developer.apple.com/standards/qtff-2001.pdf)

4
format/mp4/testdata/hevc.fqtest vendored
View File

@ -196,7 +196,7 @@ $ fq -d mp4 dv hevc.mp4
| | | boxes[0:1]: 0xa2a-0x13dc.7 (2483)
| | | [0]{}: box 0xa2a-0x13dc.7 (2483)
0x0a20| 00 00 09 b3 | .... | size: 2483 0xa2a-0xa2d.7 (4)
0x0a20| 68 65| he| type: "hev1" (High Efficiency Video Coding / H.265 / MPEG-H Part 2) 0xa2e-0xa31.7 (4)
0x0a20| 68 65| he| type: "hev1" (High Efficiency Video Coding) 0xa2e-0xa31.7 (4)
0x0a30|76 31 |v1 |
0x0a30| 00 00 00 00 00 00 | ...... | reserved: raw bits 0xa32-0xa37.7 (6)
0x0a30| 00 01 | .. | data_reference_index: 1 0xa38-0xa39.7 (2)
@ -636,4 +636,4 @@ $ fq -d mp4 dv hevc.mp4
0x0040|fd a9 78 83 ff fb 75 6c 0b 3f ff 94 ce 7f aa fe|..x...ul.?......|
* |until 0x880.7 (2127) | |
| | | id: 1 0x149b-NA (0)
| | | data_foramt: "hev1" (High Efficiency Video Coding / H.265 / MPEG-H Part 2) 0x149b-NA (0)
| | | data_format: "hev1" (High Efficiency Video Coding) 0x149b-NA (0)

2
format/mp4/testdata/in24.fqtest vendored
View File

@ -280,4 +280,4 @@ $ fq dv in24.mp4
| | | tracks[0:1]: 0x3ec-NA (0)
| | | [0]{}: track 0x3ec-NA (0)
| | | id: 1 0x3ec-NA (0)
| | | data_foramt: "in24" 0x3ec-NA (0)
| | | data_format: "in24" 0x3ec-NA (0)

2
format/mp4/testdata/lpcm.fqtest vendored
View File

@ -271,4 +271,4 @@ $ fq dv lpcm.mp4
| | | tracks[0:1]: 0x512-NA (0)
| | | [0]{}: track 0x512-NA (0)
| | | id: 1 0x512-NA (0)
| | | data_foramt: "lpcm" 0x512-NA (0)
| | | data_format: "lpcm" 0x512-NA (0)

4
format/mp4/testdata/mp3.fqtest vendored
View File

@ -192,7 +192,7 @@ $ fq -d mp4 dv mp3.mp4
| | | boxes[0:1]: 0x443-0x492.7 (80)
| | | [0]{}: box 0x443-0x492.7 (80)
0x440| 00 00 00 50 | ...P | size: 80 0x443-0x446.7 (4)
0x440| 6d 70 34 61 | mp4a | type: "mp4a" (MPEG Audio) 0x447-0x44a.7 (4)
0x440| 6d 70 34 61 | mp4a | type: "mp4a" (MPEG-4 Audio) 0x447-0x44a.7 (4)
0x440| 00 00 00 00 00| .....| reserved: raw bits 0x44b-0x450.7 (6)
0x450|00 |. |
0x450| 00 01 | .. | data_reference_index: 1 0x451-0x452.7 (2)
@ -506,4 +506,4 @@ $ fq -d mp4 dv mp3.mp4
* |until 0x29d.7 (188) | |
| | | crc_calculated: "c36b" (raw bits) 0x29e-NA (0)
| | | id: 1 0x565-NA (0)
| | | data_foramt: "mp4a" (MPEG Audio) 0x565-NA (0)
| | | data_format: "mp4a" (MPEG-4 Audio) 0x565-NA (0)

Some files were not shown because too many files have changed in this diff Show More