Summary:
Fix subtreeiter_create() to avoid calling the `SubtreeIterator` destructor on
an object that it never constructed.
If an error is thrown constructing the `cmpNodes` vector or calling the
`SubtreeIterator` constructor, then the `SubtreeIterator` destructor should
not be called. However, the `subtreeiter_create()` previously used
`Py_DECREF` to clean up the object, which always invoked
`subtreeiter_dealloc()` and called the `SubtreeIterator` destructor.
It looks like a lot of the other python object types declared in this file
have similar problems, and may incorrectly invoke C++ destructors on objects
that were never constructed if something goes wrong during object
initialization. I have not fixed the other types yet as part of this diff.
The initialization logic for each type looks somewhat different, so they will
each need different fixes rather than this simple change of just calling
`PyObject_Del()` rather than `Py_DECREF()`.
Reviewed By: quark-zju
Differential Revision: D8963858
fbshipit-source-id: d610cac16133a7ff2350c17b13b5ebc7b7becc29
Summary:
Now that all the business logic is reusable, let's create a python
datapack wrapper. In a future diff we'll allow Mercurial to instantiate this
instead of the existing cdatapack and datapack classes.
Reviewed By: quark-zju
Differential Revision: D8774486
fbshipit-source-id: 0df8ec0cc4ae7ebfc380c496b85555c78d5da505
Summary:
In a future diff we'll be add a python datapack wrapper, in addition to
the existing datastore wrapper. The logic inside them is basically identical, so
let's move all the existing logic out to a trait that can wrap all DataStore implementations.
Reviewed By: quark-zju
Differential Revision: D8774257
fbshipit-source-id: ecb9a5d174decb803c1b1216db2d366b5215ab89
Summary:
Common library to interact with mysql. Will be used in the next diffs to record
pushrebase requests
Reviewed By: quark-zju
Differential Revision: D8801603
fbshipit-source-id: 1a926742cc40458e8f9da385620f0f3837f3ae09
Summary:
A recent change made delta base's be Option types, and when we
encounter a None we treat that as the end of a delta chain. In the python
translation layer code we turned this into a Key of ("", nullid) when it
should've been (filename, nullid). This broke a test.
Reviewed By: phillco, quark-zju
Differential Revision: D8768592
fbshipit-source-id: 858a8eadad8699d2e7e99040486f836710381a4f
Summary:
The recent change from delta.base being a Key to an Option<Key> needed
to update pyrevisionstore as well. I'll add this to buck build soon so we can
catch this.
Reviewed By: phillco, quark-zju
Differential Revision: D8752910
fbshipit-source-id: ecf6d2a8a74b72d10b6dc0fb24c7b9346f0f7ed8
Summary:
Previous code format attempt (D8173629) didn't cover all files due to `**/*.py`
was not expanded recursively by bash. That makes certain changes larger than
they should be (ex. D8675439). Now use zsh's `**/*.py` to format them.
Also fix Python syntax so black can run on more files, and all lint issues.
Reviewed By: phillco
Differential Revision: D8696912
fbshipit-source-id: 95f07aa0c5eb1b63947b0f77f534957f4ab65364
Summary:
In a future diff we'll be returning data read from a pack file out as a
Delta. To avoid copies, we need to be able to return an Rc from DataPack. This
seems like it will be a common pattern, so let's go ahead and make Delta contain
its data as an Rc.
Reviewed By: quark-zju
Differential Revision: D8557949
fbshipit-source-id: 276005360bfa48e9154143dedce579a21129e976
Summary:
In a future diff we'll be serializing and deserializing metadata in
datapacks. Let's add the reader and writer functions for Metadata and some unit
tests.
Reviewed By: quark-zju
Differential Revision: D8303603
fbshipit-source-id: 7e7a7aa218c05179b205abf8b151b1488be674b3
Summary:
The windows compiler doesn't support this, so let's just hard code our
default value.
Reviewed By: singhsrb
Differential Revision: D8679518
fbshipit-source-id: 031a6e06d12c4980756e60cdeb76b6828ed8f775
Summary:
We're experiencing cases where users are pulling down far more trees
than they need. Let's enable the ability to download a certain depth of trees.
In this diff we just implement the actual iteration limiting in native code. In
a future diff we'll make the appropriate python changes and add a test.
Reviewed By: quark-zju
Differential Revision: D8523215
fbshipit-source-id: ccfeadee1525b86b1a5578631dbafe666de41f4c
Summary:
provide sockpath in the config, rather than calling `watchman get-sockname` command from python.
the config variable was already deployed on all mac
this improves hg st by 250-300 ms
(the logic gives more priority when env var if specified (the env var override is usually used for testing))
the logic has a retry mechanism if we can't connect to the socket path specified in the config
next step: deploy config on dev servers (but we will have only few ms speed up there)
we use %i (i for identity) notation for user
[298] → hg conf | grep sockpath
fsmonitor.sockpath=/opt/facebook/watchman/var/run/watchman/%i-state/sock
Reviewed By: farnz
Differential Revision: D8549130
fbshipit-source-id: 31df9ebf3eb5133d3e6f64794022cda1b0fadec4
Summary:
D8410266 introduces the use of `chrono_literals` namespace, which requires
C++14. Since we don't use C++14 otherwise, let's not use it here as well.
Reviewed By: farnz
Differential Revision: D8418398
fbshipit-source-id: c2243c2cdf51587cda698bf7c2e96691cea2d305
Summary:
This fixes how DatapackStore::refresh() computes if it needs to perform a
refresh.
Previously this code was using `clock()`, which returns the amount of CPU time
used by the program. This seems like an undesirable metric to use: the amount
of CPU used by the current program doesn't seem like it should be relevant to
whether the files on disk need to be re-examined or not. If the program has
slept for an hour we probably still want to refresh the files even though no
CPU time has been consumed.
As a result of how the computation was performed, markForRefresh() also would
not trigger a refresh if the program has used less than 100ms of CPU time so
far.
This was breaking some of the eden tests which create a repository pull some
packs into it, and then immediately refresh the store. If all of this happens
in less than 100ms the DatapackStore code would not actually refresh the
store.
Reviewed By: chadaustin
Differential Revision: D8410266
fbshipit-source-id: f08d317b261ccfe89cff45bf3e511a2b87cf9fb5
Summary:
Add a new function to `cstore.treemanifest` to list the contents of a directory
in the tree manifest.
Reviewed By: ryanmce
Differential Revision: D8332425
fbshipit-source-id: db74f11a691cd6b8ac7d51aec6b7da8149c22027
Summary:
They're actively fighting against the clang-format config
and don't have an auto-fix.
Reviewed By: quark-zju
Differential Revision: D8283622
fbshipit-source-id: 2de45f50e6370a5ed14915c6ff23dc843ff14e8a
Summary:
Optimized builds were giving a warning about shadowed variables, which
broke some of our continuous builds. Let's fix those.
Reviewed By: phillco
Differential Revision: D8287865
fbshipit-source-id: 8a2d023be93499ed6458a35ba8e8cb4d83d59e4a
Summary:
In a future diff we'll be adding the ability to request just a
subsection of a tree. In order to do that, we need to expose an api that can
iterate of just a subsection of the tree.
Since the treemanifest object itself acts mostly like a dictionary, and doesn't
expose sub-tree functionality, we expose sub-tree walks as a limited use class
level function.
Reviewed By: quark-zju
Differential Revision: D8216726
fbshipit-source-id: 0d72ef041fd6a973387f975b21e55c82f7ec789c
Summary:
A recent change renamed the methods assuming they were rust methods
but these methods are used in the python code and therefore, already had the
correct naming. This broke the one of the python tests calling the methods
which will pass now.
Reviewed By: DurhamG, quark-zju
Differential Revision: D8279742
fbshipit-source-id: 01761ad033c6669a0ae54dc1beb853a0d56df27d
Summary:
In a future diff we'll be adding the ability to iterate over just a
subsection of a tree. In order for the resulting paths to be correct, we need to
be able to prime the start path. To do so, let's add a path argument to the
SubTreeIterator, and set it to empty string everywhere it's currently used.
Reviewed By: quark-zju
Differential Revision: D8216727
fbshipit-source-id: 1969f380f7ad13f54a7dbcd2283b0428c4480ab4
Summary:
In a future diff we'll be adding a function that also needs to turn a
python store into a ManifestFetcher. So let's move that logic to a separate
function.
Reviewed By: quark-zju
Differential Revision: D8216728
fbshipit-source-id: f6d2e9577eccfcb015797a519264bd3b241ebae5
Summary:
Now that the subtree iterator doesn't contain a reference to the
treemanifest directly (it instead holds a ManifestPtr reference), we no longer
need to persist a reference to it at the python layer.
Reviewed By: quark-zju
Differential Revision: D8206301
fbshipit-source-id: 531d4399f0f54a47e0eb741f419e6242188a83ad
Summary:
Now that markPermanent no longer requires a ManifestEntry to update the
node, the resultEntry field on all the iterators is unused. Let's delete it.
Reviewed By: quark-zju
Differential Revision: D8156808
fbshipit-source-id: 21aeb050c91552c7e8973a33dd7e347457125a64
Summary:
Now that the node field is accessed via a getter that can lazily
compute it, let's get rid of the logic that proactively sets it. This will make
it easier to refactor the subtree iterator in a later diff.
Reviewed By: quark-zju
Differential Revision: D8156809
fbshipit-source-id: aabc3f68cc41baea0c166b9a2bd68bea6b1a6a03
Summary:
Previously the ManifestEntry node field was public and consumers could
read it directly. This required that the node field be updated manually, which
added complications to other code paths. In a future diff we'll be simplifying
the SubTreeIterator code to not require setting node, and to do so we make node
calculated in this accessor.
Reviewed By: quark-zju
Differential Revision: D8156807
fbshipit-source-id: e1908b2149bc0ec8fb1279e37b8a66988ff125c6
Summary:
This switches the treemanifest native iterators to use the ManifestPtr
class instead of Manifest*. ManifestPtr does ref counting, and in a future diff
we'll use this ability to have the subtree iterator iterate over a portion of a
larger tree so we can serve just parts of trees to gettreepack requests.
Reviewed By: quark-zju
Differential Revision: D8156806
fbshipit-source-id: 4dbb60d008ac8d0c789c3f2db8f7e567c5869539
Summary: This is just a refactor to address the naming scheme.
Reviewed By: quark-zju
Differential Revision: D8269217
fbshipit-source-id: 8c52d2c67837550e0b7dc1a45b3faf9a80319b61
Summary:
1. Enabled a number of additional C++ compiler warnings in Eden.
2. Fixed warnings-turned-errors that resulted from this change.
Reviewed By: simpkins
Differential Revision: D8132543
fbshipit-source-id: 2290ffaaab55024d582e29201a1bcaa1152e6b3e
Summary:
Previously we weren't including p2 when creating trees, this resulted
in incorrect conversions.
Reviewed By: phillco
Differential Revision: D8152253
fbshipit-source-id: c4f8c79b40532c5162b15032962fbc6a78d44b5f
Summary:
The memcmp contract is that the result is >0, 0, or <0. If <0 it's not
guaranteed to be -1, so let's broaden the check a bit.
Reviewed By: phillco
Differential Revision: D8190843
fbshipit-source-id: 78b864a639851a041c1a393f21a979d13eaa2e9c
Summary: Mostly empty lines removed and added. A few bugfixes on excessive line splitting.
Reviewed By: quark-zju
Differential Revision: D8199128
fbshipit-source-id: 90c1616061bfd7cfbba0b75f03f89683340374d5
Summary: Added const and override where appropriate to some classes in scm.
Reviewed By: aary
Differential Revision: D8176501
fbshipit-source-id: a9247aee569692973a822d002e951dad42c42be6
Summary:
Going to make changes to `mercurial/` for cleaner fsmonitor support
directly. So let's move the Rust python bridge there first.
Reviewed By: markbt
Differential Revision: D7909174
fbshipit-source-id: 454d784b5dca18a3af9328fc7b2f342cd4188cf6
Summary:
The method looks like a foot-gun, and it's O(all entries) instead of
O(cached entries). Change `get_tracked_filtered_key` to take an identity of
the filter function explicitly to solve the problem.
Reviewed By: markbt
Differential Revision: D7861765
fbshipit-source-id: a57ca4a7597120a5b00c63f3f373a62e19e5a834
Summary:
It's super-useful to know the difference between "Accepted (You need to go and click the land button)" and "Accepted (We're just waiting for the land system to do its thing, no work for you right now)"
Alternative approach in D8074898
Reviewed By: mitrandir77
Differential Revision: D8074794
fbshipit-source-id: fdfda64e3a542518b8609e3a415d8cb3156373cf
Summary:
This splits out the logic in the `fsmonitor` extension that is responsible for
publishing `hg.filemerge` and `hg.update` state changes to Watchman into
its own extension, `hgevents`. This is because we want the behavior of
`hgevents` when Hg is running in Eden, but we do not want the remaining
behavior of `fsmonitor` when Hg is running in Eden, so splitting the logic
into separate extensions is the most straightforward way to achieve that.
To achieve the split, we move some more logic that is common to both
`fsmonitor` and `hgevents` out of `hgext/fsmonitor/__init__.py` and into
`hgext/extlib/watchmanclient/__init__.py`. Then we move these lines
out of `extsetup()` in `fsmonitor` to create `extsetup()` in `hgevents`:
```
extensions.wrapfunction(merge, 'update', wrapupdate)
extensions.wrapfunction(filemerge, '_xmerge', _xmerge)
```
We also have to pull all of the transitive dependencies for this logic
into `hgevents`.
Finally, we also have to define a `reposetup()` function in `hgevents`
that does a subset of what `reposetup()` does in `fsmonitor`. Specifically,
it ensures that a Watchman client is created for a `repo`, as appropriate,
so that it can be used to dispatch state changes to Watchman in
`state_filemerge` and `state_update`.
Note that the utility functions `createclientforrepo()` and
`getclientforrepo()` have been added to ensure that only one
Watchman client is created (and shared) when both `fsmonitor`
and `hgevents` are enabled.
Today, when an Hg repo is created in Eden, we set `extensions.fsmonitor=!`
in the `.hg/hgrc`:
diffusion/FBS/browse/master/fbcode/eden/hooks/hg/post-clone.py$69
Therefore, to get existing repos (both Eden and non-Eden) to pick up
the `hgevents` extension automatically, we add it to the list of
`[extensions]` in `common.rc`:
diffusion/FBS/browse/master/fbcode/scm/hg/fb/staticfiles/etc/mercurial/repo-specific/common.rc$53-60
as this is where `fsmonitor` is configured. We do not enable it in
`scm/hg/fb/staticfiles/etc/mercurial/facebook.rc` because
there is no reason to enable `hgevents` on Hg servers. Therefore, we
also decline to add `hgevents` to the set of `DEFAULT_EXTENSIONS` in
`scm/hg/mercurial/extensions.py`.
Reviewed By: quark-zju
Differential Revision: D8003628
fbshipit-source-id: 4f23881f8c25f4638f5475c292537b0352ae8d15
Summary:
There is some logic in `hgext/fsmonitor/watchmanclient.py` that needs
to be shared between fsmonitor and the new extension we are trying to
split out of it as part of T29379329.
Reviewed By: quark-zju
Differential Revision: D8003255
fbshipit-source-id: de01c5ba1460d7dde22969980b296afb3a942dd7
Summary:
This is a precursor to splitting the fsmonitor extension, as both
it and the new extension will use pywatchman.
Reviewed By: quark-zju
Differential Revision: D8002713
fbshipit-source-id: 37983fe2898d23223d1178eb3f15685f17ff8868
Summary:
Initial implementation of getmissing for a simple Rust pass through
data store. Future diffs will extend this to implement the union data store
completely in Rust.
Reviewed By: quark-zju
Differential Revision: D7632405
fbshipit-source-id: e660d33f8231410805cfaba6d77c56f27b002f8e
Summary:
An initial implementation of getmeta for the Rust hg data store. Future
diffs will add more functionality.
Reviewed By: quark-zju
Differential Revision: D7632404
fbshipit-source-id: 53bd3b96b777bc3c5aef2b4d07ce1a9d9a5a52ed
Summary:
An initial implementation of getdeltachain for a simple pass through
data store. Future diffs will add additional functionality.
Reviewed By: quark-zju
Differential Revision: D7632407
fbshipit-source-id: 1a38089ba8ea70f8772af95afd871ee493082d80
Summary:
Implements the get function for a simple pass through rust data store
layer. Future diffs will implement more functions, and then later we will
implement the entire union data store in Rust.
Reviewed By: quark-zju
Differential Revision: D7632403
fbshipit-source-id: 3a1d0a8500e3110213d70dc1cff637cf8eadd809
Summary:
This will contain all the Python centric hg store code that will let
Python call into the Rust storage layer.
Reviewed By: quark-zju
Differential Revision: D7632406
fbshipit-source-id: 6b7bcc8f47a23e9c0121e1f92de1137369bf584e
Summary:
This makes it easier to modify and test the core logic without coupling with
the Python logic.
Reviewed By: markbt
Differential Revision: D7734012
fbshipit-source-id: 0d7b19198d85f6ca7314611256e9271be60070d1
Summary:
They cause a lot of churn when updating vendored crates. It's unnecessary to
check them in because the vendored crates basically pinned versions of all
dependencies.
Also clean up .gitignore, since it does not support "syntax:re".
Reviewed By: DurhamG
Differential Revision: D7562533
fbshipit-source-id: b9fcade6f38c0652a1b19ac4f90827dac7eed181
Summary:
The changes in this diff changes string comparisons using the compare method to
using equality operators.
Motivation:
- readability, simplifies code
- compare method is intended for sorting functions
This is clang check used: [Link Here](https://clang.llvm.org/extra/clang-tidy/checks/readability-string-compare.html)
Reviewed By: yfeldblum
Differential Revision: D7674416
fbshipit-source-id: dce375fb4b401f0e012ad4fc99b5dc4babe07f89
