This gives us more context about what pack file is problematic. It's useful
to detect issues like mmapping a 0-sized file.
Test Plan:
Make sure the code compiles without warnings. Run related tests.
```
make local
rt test-cstore.t
```
Make sure `setup.py` specifies C99 so variable length array is available.
Differential Revision: https://phab.mercurial-scm.org/D1125
Summary:
Clean up the include guards to be more consistent and unique.
Some files used include guards like "KEY_H" and "STORE_H" which were not very
unique, and are more likely to collide with definitions provided by header
files from other projects. Some of the py-*.h files were missing include
guards altogether.
This corresponds to Facebook diff D5588670.
Test Plan: Confirmed the code still builds and passes tests.
Reviewers: #fbhgext, quark
Reviewed By: #fbhgext, quark
Differential Revision: https://phab.mercurial-scm.org/D508
Summary:
Update the copyright headers in most of the C/C++ code consistently use the
GPLv2 copyright message. This allows these files to pass Facebook's internal
C/C++ linters.
Some of the files in fbcode/scm/hgext/cstore/ appear to have actually been
copied from the hg-crew repository, and were not originally authored by
Facebook. I have not modified the copyright statements in these files:
- cstore/bitmanipulation.h
- cstore/compat.h
- cstore/mpatch.h
- cstore/mpatch.c
I also have not modified any of the cfastmanifest code.
This corresponds to Facebook diff D5588677.
Test Plan:
Confirmed that Facebook's C++ linters no longer complain about the copyright
messages.
Reviewers: #fbhgext, quark
Reviewed By: #fbhgext, quark
Differential Revision: https://phab.mercurial-scm.org/D507
Summary:
Convert the C and C++ files in cdatapack, clib, cstore, and ctreemanifest
to always include files from the root of fb-hgext. This simplifies the build
process by no long requiring a variety of separate include directories to be
specified on the compiler command line.
This will also make it easier to re-use these extensions in other projects
with different build systems.
This corresponds to the Facebook diff D5588676.
Test Plan: Confirmed that "make local" succeeds from a clean build.
Reviewers: #fbhgext, quark
Reviewed By: #fbhgext, quark
Differential Revision: https://phab.mercurial-scm.org/D505
Summary:
This patch removes all `#include "../` lines and use the shortest possible
include path.
Test Plan: `make clean build`
Reviewers: durham, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5113672
Signature: t1:5113672:1495565454:961fb6f2f57a81a95013e0b8f67b2917c2e4523e
Summary:
This diff makes 2 changes to v1 packfile metadata:
1. Move `key` in a metadata entry to before `size`.
```
old: [entry-size: 2 byte] [key: 1 byte] [data: var length]
new: [key: 1 byte] [data-size: 2 byte] [data: var length]
```
Previously `entry-size == 0` does not make sense.
2. Use binary to represent sizes, instead of ASCII.
Related utility methods are cleaned up a bit so it's harder to make mistakes.
Test Plan: Updated existing tests
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4983189
Signature: t1:4983189:1493689852:22d544d73ed63fac83f849786de035af304161ce
Summary:
This diff implements getmeta in C and enables related tests.
Now all content stores support `getmeta`.
Test Plan: Run existing tests.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4960926
Signature: t1:4960926:1493611048:55095c32927fac74e698f21d47173cb8a7523fb6
Summary:
This diffs add a `getmeta` method to all content stores. The cdatapack code is
modified to pass the tests, it needs further change to support `getmeta`.
The datapack format is bumped to v1 from v0. For v1, we append a `metadata`
dict at the end of each revision. The dict is currently used to store revlog
flags and rawsize of raw revlog fulltext. In the future we can put more data
like a second hash etc, without changing API or format again.
This diff focuses on correctness. A datapack caching layer to speed up
`getmeta` will be added later.
Tests are updated since we write new v1 packfile now and the format change
leads to different content and packfile names.
`Makefile`, `ls-l.py` are added to make tests easier to maintain.
Test Plan: Updated existing tests.
Reviewers: #mercurial, rmcelroy, durham
Reviewed By: durham
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4903917
Signature: t1:4903917:1493255844:7ef5d487096cd2f78f2aaae672a68d49f33632ee
Summary:
This is an RFC kind of diff, so I am looking for comments. I am not very
familiar with either C or C++.
The goal is to make all this stuff compile on Windows under MSVC2015. MSVC
does not support variable-sized arrays in C++ out-of-the-box (not sure
whether there are tricks to make it do so), so my proposal is to use
explicit `malloc`s in place.
Test Plan:
Only tested on Linux for now. `python setup.py build` works,
tests pass, seems to make sense.
Reviewers: #sourcecontrol, jsgf
Reviewed By: jsgf
Subscribers: jsgf, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4839968
Signature: t1:4839968:1491492552:c80fac6990aaee78e6bb18522ff13e02eb9521e4
Summary:
The remaining python parts of the store are a perf bottleneck when accessing
hundreds of thousands of pack file entries (like in treemanifest). Let's
implement them in C++.
This first patch just add the basic boiler plate, and implements a single
function getdeltachain(), with a test. Future patches will add more
functionality and other parts of the store.
Since cstore depends on cdatapack and ctreemanifest (the pythonutils.h part for
now), we need to tweak our setup.py to enforce a certain build order too.
Test Plan: Added a test, yo
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4547929
Signature: t1:4547929:1487181318:21c146cf370d26cb97efe6a883868b85b4e32f49
Summary:
As part of unifying our storage layer into a single library, let's move
py-cdatapack into the new cstore directory. Future patches will move
ctreemanifest and the upcoming datapackstore into here as well.
py-cdatapack.h required some reordering since it seems forward declarations work
a little differently between C and C++. There were no code changes though,
except one int->size_t fix.
Test Plan: Ran the tests
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4581320
Signature: t1:4581320:1487788968:e8a34c7a03a16db282214c7dd476b749b92a1bfa
