Summary:
The `Node` type will be used in multiple places. Let's move it to a standalone
crate so new libraries depending on it won't need to pull in all of
revisionstore's dependencies.
Note: I'd also like the `types` create to only define clean types. Given the
fact NULL_ID is not a great design in Mercurial (`Option<Node>` is a better
choice in Rust), it probably does not belong to the formal Rust `Node` type.
This diff is merely about moving things with minimal changes. NULL_ID will
be decoupled from `Node` in a follow-up.
Reviewed By: markbt
Differential Revision: D10132047
fbshipit-source-id: 5d05c5e0ac06a2d58556c4db11775503f9495626
Summary:
as it states in the comments, we ideally wouldn't get
in this state, but if we do, let's try to recover and continue
a bit more gracefully.
Reviewed By: quark-zju
Differential Revision: D9445209
fbshipit-source-id: a08e605f37f85a9296f71d595f950bbf110e204d
Summary:
Sync up with pywatchman changes; some of these are rather
old now :-/
Reviewed By: quark-zju
Differential Revision: D9440040
fbshipit-source-id: a19b53bb21ffcb8e2566bfa7461a636bf1b739aa
Summary:
In the later diffs I'll add some more functionality there, not strictly
related to encoding paths.
Reviewed By: quark-zju
Differential Revision: D9441427
fbshipit-source-id: 069ab30a24761038fa2c1a4f180bbc0699d38ef9
Summary:
The python getancestors implementation expects a known optional
argument. Unfortunately I can't get rust cpython to support optional arguments,
so for now we make it required.
Reviewed By: singhsrb
Differential Revision: D9367480
fbshipit-source-id: 47dd74e6cbd4fe5a774ebf0949e2114bae15844d
Summary:
For some arcrc formats, hg commands prefer a deprecated blob in the .arcrc file which does not contain an oauth token, and only contains a phabricator cert (which is deprecated in favor of oauth)
This updates the logic to look for the phabricator.intern.facebook.com blob in the .arcrc to get an oauth token.
Reviewed By: mitrandir77
Differential Revision: D9351851
fbshipit-source-id: 11efe30b077480caa7b147c45a8d0eb78189277e
Summary:
Adds the cpython bindings that will let us call HistoryPack from rust
code.
Reviewed By: markbt
Differential Revision: D9273267
fbshipit-source-id: a29c97607d428255784c6508773cf14e283f287c
Summary:
Implements the python get_node_info function on top of an existing
HistoryStore.
Reviewed By: markbt
Differential Revision: D9273266
fbshipit-source-id: c29ce2636394a6f46fc470508d8c789873850f74
Summary:
Implements the python get_missing function on top of an existing
HistoryStore.
Reviewed By: markbt
Differential Revision: D9273262
fbshipit-source-id: ca6e123e670a57e740df83fbcc0a49582a6be3e2
Summary:
Implements the python get_ancestors function on top of an existing
HistoryStore.
Reviewed By: markbt
Differential Revision: D9273263
fbshipit-source-id: 7c369a7681161cc93ffdda860081ae306ce3e06b
Summary:
Declares the HistoryStorePyExt trait and implements it for any class
that implements HistoryStore. In future diffs we'll implement each of the
functions, then call the functions from a rust cpython declared historypack
class.
Reviewed By: markbt
Differential Revision: D9273265
fbshipit-source-id: 8f2dabc54072076600a5cc22a814485df9035211
Summary:
In an upcoming diff we'll be adding HistoryStore python extension
methods, so let's move all the existing ones to their own files to keep things
clean.
Reviewed By: markbt
Differential Revision: D9273268
fbshipit-source-id: 56be3e3130c2c95d80510915d0d1a14e2fddc783
Summary:
Let's use the pathencoding crate instead of a lossy conversion to
strings.
Reviewed By: quark-zju
Differential Revision: D8992738
fbshipit-source-id: 2975b421a3de5d7fafb6688260d9482af701092e
Summary:
The paths for each data pack are used in various situations (repack,
error reporting, etc) so let's store them and make them accessible via the
python api.
Reviewed By: quark-zju
Differential Revision: D8884773
fbshipit-source-id: 4108c98b4e303ba9bded1f264746fa4a84845c73
Summary:
For some reason, buck cython build starts to treat unprefixed strings as
unicode strings. That breaks Mercurial where everything should be bytes.
Therefore add the b prefix to make them byte strings explicitly.
Reviewed By: DurhamG, ryanmce
Differential Revision: D9015176
fbshipit-source-id: 7a3ef9d735362e3ae02208ec19d0551bf0adaaa4
Summary:
Previously it would only accept lists. This was fine for remotefilelog,
but treemanifest actually passes generators here, so we need to change this to
accept an iterator.
Reviewed By: quark-zju
Differential Revision: D8813143
fbshipit-source-id: 1d0cbf32d4c56521e26c55a6a061c22a1e618aed
Summary:
Previously we converted all rust errors into python KeyErrors. Let's
cast KeyErrors into python KeyErrors and everything else into RuntimeErrors for
now. We can expand this as needed.
Reviewed By: farnz
Differential Revision: D8775753
fbshipit-source-id: f72e96db752167110d989226f2dedb9ff5bd82a3
Summary:
This function is present on the python data store api, so let's
replicate it here. Later we should come back and refactor this to be a special
case of the get_delta_chain result, but for now we'll maintain the custom API so
we can start using this code from python.
Reviewed By: quark-zju
Differential Revision: D8774474
fbshipit-source-id: aabcff3a43ae68859a1bf3b23f433214571b1a9d
Summary:
Fix subtreeiter_create() to avoid calling the `SubtreeIterator` destructor on
an object that it never constructed.
If an error is thrown constructing the `cmpNodes` vector or calling the
`SubtreeIterator` constructor, then the `SubtreeIterator` destructor should
not be called. However, the `subtreeiter_create()` previously used
`Py_DECREF` to clean up the object, which always invoked
`subtreeiter_dealloc()` and called the `SubtreeIterator` destructor.
It looks like a lot of the other python object types declared in this file
have similar problems, and may incorrectly invoke C++ destructors on objects
that were never constructed if something goes wrong during object
initialization. I have not fixed the other types yet as part of this diff.
The initialization logic for each type looks somewhat different, so they will
each need different fixes rather than this simple change of just calling
`PyObject_Del()` rather than `Py_DECREF()`.
Reviewed By: quark-zju
Differential Revision: D8963858
fbshipit-source-id: d610cac16133a7ff2350c17b13b5ebc7b7becc29
Summary:
Now that all the business logic is reusable, let's create a python
datapack wrapper. In a future diff we'll allow Mercurial to instantiate this
instead of the existing cdatapack and datapack classes.
Reviewed By: quark-zju
Differential Revision: D8774486
fbshipit-source-id: 0df8ec0cc4ae7ebfc380c496b85555c78d5da505
Summary:
In a future diff we'll be add a python datapack wrapper, in addition to
the existing datastore wrapper. The logic inside them is basically identical, so
let's move all the existing logic out to a trait that can wrap all DataStore implementations.
Reviewed By: quark-zju
Differential Revision: D8774257
fbshipit-source-id: ecb9a5d174decb803c1b1216db2d366b5215ab89
Summary:
Common library to interact with mysql. Will be used in the next diffs to record
pushrebase requests
Reviewed By: quark-zju
Differential Revision: D8801603
fbshipit-source-id: 1a926742cc40458e8f9da385620f0f3837f3ae09
Summary:
A recent change made delta base's be Option types, and when we
encounter a None we treat that as the end of a delta chain. In the python
translation layer code we turned this into a Key of ("", nullid) when it
should've been (filename, nullid). This broke a test.
Reviewed By: phillco, quark-zju
Differential Revision: D8768592
fbshipit-source-id: 858a8eadad8699d2e7e99040486f836710381a4f
Summary:
The recent change from delta.base being a Key to an Option<Key> needed
to update pyrevisionstore as well. I'll add this to buck build soon so we can
catch this.
Reviewed By: phillco, quark-zju
Differential Revision: D8752910
fbshipit-source-id: ecf6d2a8a74b72d10b6dc0fb24c7b9346f0f7ed8
Summary:
Previous code format attempt (D8173629) didn't cover all files due to `**/*.py`
was not expanded recursively by bash. That makes certain changes larger than
they should be (ex. D8675439). Now use zsh's `**/*.py` to format them.
Also fix Python syntax so black can run on more files, and all lint issues.
Reviewed By: phillco
Differential Revision: D8696912
fbshipit-source-id: 95f07aa0c5eb1b63947b0f77f534957f4ab65364
Summary:
In a future diff we'll be returning data read from a pack file out as a
Delta. To avoid copies, we need to be able to return an Rc from DataPack. This
seems like it will be a common pattern, so let's go ahead and make Delta contain
its data as an Rc.
Reviewed By: quark-zju
Differential Revision: D8557949
fbshipit-source-id: 276005360bfa48e9154143dedce579a21129e976
Summary:
In a future diff we'll be serializing and deserializing metadata in
datapacks. Let's add the reader and writer functions for Metadata and some unit
tests.
Reviewed By: quark-zju
Differential Revision: D8303603
fbshipit-source-id: 7e7a7aa218c05179b205abf8b151b1488be674b3
Summary:
The windows compiler doesn't support this, so let's just hard code our
default value.
Reviewed By: singhsrb
Differential Revision: D8679518
fbshipit-source-id: 031a6e06d12c4980756e60cdeb76b6828ed8f775
Summary:
We're experiencing cases where users are pulling down far more trees
than they need. Let's enable the ability to download a certain depth of trees.
In this diff we just implement the actual iteration limiting in native code. In
a future diff we'll make the appropriate python changes and add a test.
Reviewed By: quark-zju
Differential Revision: D8523215
fbshipit-source-id: ccfeadee1525b86b1a5578631dbafe666de41f4c
Summary:
provide sockpath in the config, rather than calling `watchman get-sockname` command from python.
the config variable was already deployed on all mac
this improves hg st by 250-300 ms
(the logic gives more priority when env var if specified (the env var override is usually used for testing))
the logic has a retry mechanism if we can't connect to the socket path specified in the config
next step: deploy config on dev servers (but we will have only few ms speed up there)
we use %i (i for identity) notation for user
[298] → hg conf | grep sockpath
fsmonitor.sockpath=/opt/facebook/watchman/var/run/watchman/%i-state/sock
Reviewed By: farnz
Differential Revision: D8549130
fbshipit-source-id: 31df9ebf3eb5133d3e6f64794022cda1b0fadec4
Summary:
D8410266 introduces the use of `chrono_literals` namespace, which requires
C++14. Since we don't use C++14 otherwise, let's not use it here as well.
Reviewed By: farnz
Differential Revision: D8418398
fbshipit-source-id: c2243c2cdf51587cda698bf7c2e96691cea2d305
Summary:
This fixes how DatapackStore::refresh() computes if it needs to perform a
refresh.
Previously this code was using `clock()`, which returns the amount of CPU time
used by the program. This seems like an undesirable metric to use: the amount
of CPU used by the current program doesn't seem like it should be relevant to
whether the files on disk need to be re-examined or not. If the program has
slept for an hour we probably still want to refresh the files even though no
CPU time has been consumed.
As a result of how the computation was performed, markForRefresh() also would
not trigger a refresh if the program has used less than 100ms of CPU time so
far.
This was breaking some of the eden tests which create a repository pull some
packs into it, and then immediately refresh the store. If all of this happens
in less than 100ms the DatapackStore code would not actually refresh the
store.
Reviewed By: chadaustin
Differential Revision: D8410266
fbshipit-source-id: f08d317b261ccfe89cff45bf3e511a2b87cf9fb5
Summary:
Add a new function to `cstore.treemanifest` to list the contents of a directory
in the tree manifest.
Reviewed By: ryanmce
Differential Revision: D8332425
fbshipit-source-id: db74f11a691cd6b8ac7d51aec6b7da8149c22027
Summary:
They're actively fighting against the clang-format config
and don't have an auto-fix.
Reviewed By: quark-zju
Differential Revision: D8283622
fbshipit-source-id: 2de45f50e6370a5ed14915c6ff23dc843ff14e8a
Summary:
Optimized builds were giving a warning about shadowed variables, which
broke some of our continuous builds. Let's fix those.
Reviewed By: phillco
Differential Revision: D8287865
fbshipit-source-id: 8a2d023be93499ed6458a35ba8e8cb4d83d59e4a
Summary:
In a future diff we'll be adding the ability to request just a
subsection of a tree. In order to do that, we need to expose an api that can
iterate of just a subsection of the tree.
Since the treemanifest object itself acts mostly like a dictionary, and doesn't
expose sub-tree functionality, we expose sub-tree walks as a limited use class
level function.
Reviewed By: quark-zju
Differential Revision: D8216726
fbshipit-source-id: 0d72ef041fd6a973387f975b21e55c82f7ec789c
Summary:
A recent change renamed the methods assuming they were rust methods
but these methods are used in the python code and therefore, already had the
correct naming. This broke the one of the python tests calling the methods
which will pass now.
Reviewed By: DurhamG, quark-zju
Differential Revision: D8279742
fbshipit-source-id: 01761ad033c6669a0ae54dc1beb853a0d56df27d
Summary:
In a future diff we'll be adding the ability to iterate over just a
subsection of a tree. In order for the resulting paths to be correct, we need to
be able to prime the start path. To do so, let's add a path argument to the
SubTreeIterator, and set it to empty string everywhere it's currently used.
Reviewed By: quark-zju
Differential Revision: D8216727
fbshipit-source-id: 1969f380f7ad13f54a7dbcd2283b0428c4480ab4
Summary:
In a future diff we'll be adding a function that also needs to turn a
python store into a ManifestFetcher. So let's move that logic to a separate
function.
Reviewed By: quark-zju
Differential Revision: D8216728
fbshipit-source-id: f6d2e9577eccfcb015797a519264bd3b241ebae5
Summary:
Now that the subtree iterator doesn't contain a reference to the
treemanifest directly (it instead holds a ManifestPtr reference), we no longer
need to persist a reference to it at the python layer.
Reviewed By: quark-zju
Differential Revision: D8206301
fbshipit-source-id: 531d4399f0f54a47e0eb741f419e6242188a83ad
Summary:
Now that markPermanent no longer requires a ManifestEntry to update the
node, the resultEntry field on all the iterators is unused. Let's delete it.
Reviewed By: quark-zju
Differential Revision: D8156808
fbshipit-source-id: 21aeb050c91552c7e8973a33dd7e347457125a64
Summary:
Now that the node field is accessed via a getter that can lazily
compute it, let's get rid of the logic that proactively sets it. This will make
it easier to refactor the subtree iterator in a later diff.
Reviewed By: quark-zju
Differential Revision: D8156809
fbshipit-source-id: aabc3f68cc41baea0c166b9a2bd68bea6b1a6a03
Summary:
Previously the ManifestEntry node field was public and consumers could
read it directly. This required that the node field be updated manually, which
added complications to other code paths. In a future diff we'll be simplifying
the SubTreeIterator code to not require setting node, and to do so we make node
calculated in this accessor.
Reviewed By: quark-zju
Differential Revision: D8156807
fbshipit-source-id: e1908b2149bc0ec8fb1279e37b8a66988ff125c6
Summary:
This switches the treemanifest native iterators to use the ManifestPtr
class instead of Manifest*. ManifestPtr does ref counting, and in a future diff
we'll use this ability to have the subtree iterator iterate over a portion of a
larger tree so we can serve just parts of trees to gettreepack requests.
Reviewed By: quark-zju
Differential Revision: D8156806
fbshipit-source-id: 4dbb60d008ac8d0c789c3f2db8f7e567c5869539
Summary: This is just a refactor to address the naming scheme.
Reviewed By: quark-zju
Differential Revision: D8269217
fbshipit-source-id: 8c52d2c67837550e0b7dc1a45b3faf9a80319b61
Summary:
1. Enabled a number of additional C++ compiler warnings in Eden.
2. Fixed warnings-turned-errors that resulted from this change.
Reviewed By: simpkins
Differential Revision: D8132543
fbshipit-source-id: 2290ffaaab55024d582e29201a1bcaa1152e6b3e
