Commit Graph

5 Commits

Author SHA1 Message Date
Jun Wu
488cf5ea26 cdatapack: fix empty revision reading
Summary:
The newly changed `LZ4_decompress_safe` is unhappy about empty buffer. So let's
check empty revision explicitly.

Test Plan: Added a test

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4986277

Signature: t1:4986277:1493743735:be2ad6047bb0b983425c6e49b7c5ddf2c94d1c1a
2017-05-02 10:01:13 -07:00
Jun Wu
4a613bcaef datapack: update the format about metadata
Summary:
This diff makes 2 changes to v1 packfile metadata:

1. Move `key` in a metadata entry to before `size`.

```
old: [entry-size: 2 byte] [key: 1 byte] [data: var length]
new: [key: 1 byte] [data-size: 2 byte] [data: var length]
```

Previously `entry-size == 0` does not make sense.

2. Use binary to represent sizes, instead of ASCII.

Related utility methods are cleaned up a bit so it's harder to make mistakes.

Test Plan: Updated existing tests

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4983189

Signature: t1:4983189:1493689852:22d544d73ed63fac83f849786de035af304161ce
2017-05-01 19:03:25 -07:00
Jun Wu
9a8545ebc1 cdatapack: implement getmeta for C module
Summary:
This diff implements getmeta in C and enables related tests.

Now all content stores support `getmeta`.

Test Plan: Run existing tests.

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4960926

Signature: t1:4960926:1493611048:55095c32927fac74e698f21d47173cb8a7523fb6
2017-05-01 13:29:19 -07:00
Jun Wu
4240bd017e remotefilelog: let content stores support metadata
Summary:
This diffs add a `getmeta` method to all content stores. The cdatapack code is
modified to pass the tests, it needs further change to support `getmeta`.

The datapack format is bumped to v1 from v0. For v1, we append a `metadata`
dict at the end of each revision. The dict is currently used to store revlog
flags and rawsize of raw revlog fulltext. In the future we can put more data
like a second hash etc, without changing API or format again.

This diff focuses on correctness. A datapack caching layer to speed up
`getmeta` will be added later.

Tests are updated since we write new v1 packfile now and the format change
leads to different content and packfile names.

`Makefile`, `ls-l.py` are added to make tests easier to maintain.

Test Plan: Updated existing tests.

Reviewers: #mercurial, rmcelroy, durham

Reviewed By: durham

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4903917

Signature: t1:4903917:1493255844:7ef5d487096cd2f78f2aaae672a68d49f33632ee
2017-04-26 19:50:36 -07:00
Durham Goode
85ff1268f8 cstore-tests: move remotefilelog py tests to .t test
Summary:
A future patch which renamed cdatapack.so to cstore.so pointed out that our
test-remotefilelog*.py tests are running against the system remotefilelog. Let's
move them to a .t test that sets the appropriate PYTHONPATH.

Test Plan: Ran the tests

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4581281

Signature: t1:4581281:1487788843:365c6adf41ba51bdbe7394c1ab423fcc5a63bdbc
2017-02-23 14:03:02 -08:00