Commit Graph

87 Commits

Author SHA1 Message Date
Liubov Dmitrieva
6db13f4759 fsmonitor: black format + unused import
Reviewed By: singhsrb

Differential Revision: D8551677

fbshipit-source-id: fe98ae41a382a499823bd52c518a269b5f8e3b40
2018-06-20 15:20:14 -07:00
Kostia Balytskyi
a322b60e97 datapackstore: use C++11-compatible chrono code
Summary:
D8410266 introduces the use of `chrono_literals` namespace, which requires
C++14. Since we don't use C++14 otherwise, let's not use it here as well.

Reviewed By: farnz

Differential Revision: D8418398

fbshipit-source-id: c2243c2cdf51587cda698bf7c2e96691cea2d305
2018-06-14 06:23:26 -07:00
Adam Simpkins
ce351de8fb ensure that DatapackStore::markForRefresh() actually refreshes the store
Summary:
This fixes how DatapackStore::refresh() computes if it needs to perform a
refresh.

Previously this code was using `clock()`, which returns the amount of CPU time
used by the program.  This seems like an undesirable metric to use: the amount
of CPU used by the current program doesn't seem like it should be relevant to
whether the files on disk need to be re-examined or not.  If the program has
slept for an hour we probably still want to refresh the files even though no
CPU time has been consumed.

As a result of how the computation was performed, markForRefresh() also would
not trigger a refresh if the program has used less than 100ms of CPU time so
far.

This was breaking some of the eden tests which create a repository pull some
packs into it, and then immediately refresh the store.  If all of this happens
in less than 100ms the DatapackStore code would not actually refresh the
store.

Reviewed By: chadaustin

Differential Revision: D8410266

fbshipit-source-id: f08d317b261ccfe89cff45bf3e511a2b87cf9fb5
2018-06-13 19:39:12 -07:00
Mark Thomas
af99203255 cstore: add treemanifest.listdir
Summary:
Add a new function to `cstore.treemanifest` to list the contents of a directory
in the tree manifest.

Reviewed By: ryanmce

Differential Revision: D8332425

fbshipit-source-id: db74f11a691cd6b8ac7d51aec6b7da8149c22027
2018-06-12 09:23:44 -07:00
Wez Furlong
31bcfbe58e hg: disable check-code tests for C code
Summary:
They're actively fighting against the clang-format config
and don't have an auto-fix.

Reviewed By: quark-zju

Differential Revision: D8283622

fbshipit-source-id: 2de45f50e6370a5ed14915c6ff23dc843ff14e8a
2018-06-05 19:21:43 -07:00
Durham Goode
c244d056b9 treemanifest: auto-format code
Summary: This fixes various clang formatter lint warnings.

Reviewed By: phillco, singhsrb

Differential Revision: D8283573

fbshipit-source-id: c466ccaf8d5e50a7421fd34e43cf49559431d416
2018-06-05 16:40:44 -07:00
Durham Goode
6bad025052 treemanifest: fix build warning about shadowed variables
Summary:
Optimized builds were giving a warning about shadowed variables, which
broke some of our continuous builds. Let's fix those.

Reviewed By: phillco

Differential Revision: D8287865

fbshipit-source-id: 8a2d023be93499ed6458a35ba8e8cb4d83d59e4a
2018-06-05 14:35:59 -07:00
Durham Goode
dee59f7ffc treemanifest: add a subdir tree walking api
Summary:
In a future diff we'll be adding the ability to request just a
subsection of a tree. In order to do that, we need to expose an api that can
iterate of just a subsection of the tree.

Since the treemanifest object itself acts mostly like a dictionary, and doesn't
expose sub-tree functionality, we expose sub-tree walks as a limited use class
level function.

Reviewed By: quark-zju

Differential Revision: D8216726

fbshipit-source-id: 0d72ef041fd6a973387f975b21e55c82f7ec789c
2018-06-05 11:45:25 -07:00
Saurabh Singh
5427b362d1 fix method naming
Summary:
A recent change renamed the methods assuming they were rust methods
but these methods are used in the python code and therefore, already had the
correct naming. This broke the one of the python tests calling the methods
which will pass now.

Reviewed By: DurhamG, quark-zju

Differential Revision: D8279742

fbshipit-source-id: 01761ad033c6669a0ae54dc1beb853a0d56df27d
2018-06-05 09:08:21 -07:00
Durham Goode
1ac8657f83 treemanifest: require setting the start path for SubTreeIterator
Summary:
In a future diff we'll be adding the ability to iterate over just a
subsection of a tree. In order for the resulting paths to be correct, we need to
be able to prime the start path. To do so, let's add a path argument to the
SubTreeIterator, and set it to empty string everywhere it's currently used.

Reviewed By: quark-zju

Differential Revision: D8216727

fbshipit-source-id: 1969f380f7ad13f54a7dbcd2283b0428c4480ab4
2018-06-04 18:23:03 -07:00
Durham Goode
6370628aeb treemanifest: move store logic to a separate function
Summary:
In a future diff we'll be adding a function that also needs to turn a
python store into a ManifestFetcher. So let's move that logic to a separate
function.

Reviewed By: quark-zju

Differential Revision: D8216728

fbshipit-source-id: f6d2e9577eccfcb015797a519264bd3b241ebae5
2018-06-04 18:23:03 -07:00
Durham Goode
18945e39b6 treemanifest: remove py_treemanifest from iterators
Summary:
Now that the subtree iterator doesn't contain a reference to the
treemanifest directly (it instead holds a ManifestPtr reference), we no longer
need to persist a reference to it at the python layer.

Reviewed By: quark-zju

Differential Revision: D8206301

fbshipit-source-id: 531d4399f0f54a47e0eb741f419e6242188a83ad
2018-06-04 18:23:03 -07:00
Durham Goode
aa1e1a5604 treemanifest: remove resultEntry from iterators
Summary:
Now that markPermanent no longer requires a ManifestEntry to update the
node, the resultEntry field on all the iterators is unused. Let's delete it.

Reviewed By: quark-zju

Differential Revision: D8156808

fbshipit-source-id: 21aeb050c91552c7e8973a33dd7e347457125a64
2018-06-04 18:23:03 -07:00
Durham Goode
4708242f44 treemanifest: remove logic that updates ManifestEntry.node
Summary:
Now that the node field is accessed via a getter that can lazily
compute it, let's get rid of the logic that proactively sets it. This will make
it easier to refactor the subtree iterator in a later diff.

Reviewed By: quark-zju

Differential Revision: D8156809

fbshipit-source-id: aabc3f68cc41baea0c166b9a2bd68bea6b1a6a03
2018-06-04 18:23:03 -07:00
Durham Goode
63834775d7 treemanifest: move ManifestEntry node behind an accessor
Summary:
Previously the ManifestEntry node field was public and consumers could
read it directly. This required that the node field be updated manually, which
added complications to other code paths. In a future diff we'll be simplifying
the SubTreeIterator code to not require setting node, and to do so we make node
calculated in this accessor.

Reviewed By: quark-zju

Differential Revision: D8156807

fbshipit-source-id: e1908b2149bc0ec8fb1279e37b8a66988ff125c6
2018-06-04 18:23:03 -07:00
Durham Goode
0bbc6b6666 treemanifest: switch native iterators to use ManifestPtr
Summary:
This switches the treemanifest native iterators to use the ManifestPtr
class instead of Manifest*. ManifestPtr does ref counting, and in a future diff
we'll use this ability to have the subtree iterator iterate over a portion of a
larger tree so we can serve just parts of trees to gettreepack requests.

Reviewed By: quark-zju

Differential Revision: D8156806

fbshipit-source-id: 4dbb60d008ac8d0c789c3f2db8f7e567c5869539
2018-06-04 18:23:03 -07:00
Saurabh Singh
7c9227818a refactor rust datastore to a consistent naming scene
Summary: This is just a refactor to address the naming scheme.

Reviewed By: quark-zju

Differential Revision: D8269217

fbshipit-source-id: 8c52d2c67837550e0b7dc1a45b3faf9a80319b61
2018-06-04 17:39:47 -07:00
Sergey Zhupanov
7d44cb5f01 Enabled additional compiler warnings in Eden.
Summary:
1. Enabled a number of additional C++ compiler warnings in Eden.
2. Fixed warnings-turned-errors that resulted from this change.

Reviewed By: simpkins

Differential Revision: D8132543

fbshipit-source-id: 2290ffaaab55024d582e29201a1bcaa1152e6b3e
2018-06-01 11:10:54 -07:00
Durham Goode
1aa9db8542 treemanifest: include p2 when creating trees
Summary:
Previously we weren't including p2 when creating trees, this resulted
in incorrect conversions.

Reviewed By: phillco

Differential Revision: D8152253

fbshipit-source-id: c4f8c79b40532c5162b15032962fbc6a78d44b5f
2018-05-30 18:20:57 -07:00
Durham Goode
06e89ef39e treemanifest: check memcmp < 0 instead of memcmp == -1
Summary:
The memcmp contract is that the result is >0, 0, or <0. If <0 it's not
guaranteed to be -1, so let's broaden the check a bit.

Reviewed By: phillco

Differential Revision: D8190843

fbshipit-source-id: 78b864a639851a041c1a393f21a979d13eaa2e9c
2018-05-30 18:20:57 -07:00
Lukasz Langa
dfda82e492 Upgrade to 18.5b1
Summary: Mostly empty lines removed and added.  A few bugfixes on excessive line splitting.

Reviewed By: quark-zju

Differential Revision: D8199128

fbshipit-source-id: 90c1616061bfd7cfbba0b75f03f89683340374d5
2018-05-30 02:23:58 -07:00
Sergey Zhupanov
3288f45188 Minor cleanup of interfaces in some scm classes.
Summary: Added const and override where appropriate to some classes in scm.

Reviewed By: aary

Differential Revision: D8176501

fbshipit-source-id: a9247aee569692973a822d002e951dad42c42be6
2018-05-29 13:26:31 -07:00
Jun Wu
ee83f12849 treestate: move hgext.extlib.treedirstate to mercurial.rust.treestate
Summary:
Going to make changes to `mercurial/` for cleaner fsmonitor support
directly. So let's move the Rust python bridge there first.

Reviewed By: markbt

Differential Revision: D7909174

fbshipit-source-id: 454d784b5dca18a3af9328fc7b2f342cd4188cf6
2018-05-26 14:05:18 -07:00
Jun Wu
bfc47200de treedirstate: expose TreeState APIs
Summary: Expose `TreeState` features to the Python world.

Reviewed By: markbt

Differential Revision: D7886283

fbshipit-source-id: 39e654bfcd611aeff4255e1fff8ce32f20544a22
2018-05-26 10:50:26 -07:00
Jun Wu
b27143828b treestate: change Key from Vec<u8> to Box<[u8]>
Summary: This saves one `usize` per `Key`.

Differential Revision: D7861766

fbshipit-source-id: e44d6b98758966edd0f9823f2f50270ba5481b22
2018-05-23 06:12:45 -07:00
Jun Wu
395edeaea6 treestate: remove clear_filtered_keys
Summary:
The method looks like a foot-gun, and it's O(all entries) instead of
O(cached entries). Change `get_tracked_filtered_key` to take an identity of
the filter function explicitly to solve the problem.

Reviewed By: markbt

Differential Revision: D7861765

fbshipit-source-id: a57ca4a7597120a5b00c63f3f373a62e19e5a834
2018-05-23 06:12:45 -07:00
Shish Girling
1137cc5743 display 'Landing' status in hg ssl output
Summary:
It's super-useful to know the difference between "Accepted (You need to go and click the land button)" and "Accepted (We're just waiting for the land system to do its thing, no work for you right now)"

Alternative approach in D8074898

Reviewed By: mitrandir77

Differential Revision: D8074794

fbshipit-source-id: fdfda64e3a542518b8609e3a415d8cb3156373cf
2018-05-22 02:51:56 -07:00
Michael Bolin
a5e1820702 Split out a new hgevents extension from fsmonitor.
Summary:
This splits out the logic in the `fsmonitor` extension that is responsible for
publishing `hg.filemerge` and `hg.update` state changes to Watchman into
its own extension, `hgevents`. This is because we want the behavior of
`hgevents` when Hg is running in Eden, but we do not want the remaining
behavior of `fsmonitor` when Hg is running in Eden, so splitting the logic
into separate extensions is the most straightforward way to achieve that.

To achieve the split, we move some more logic that is common to both
`fsmonitor` and `hgevents` out of `hgext/fsmonitor/__init__.py` and into
`hgext/extlib/watchmanclient/__init__.py`. Then we move these lines
out of `extsetup()` in `fsmonitor` to create `extsetup()` in `hgevents`:

```
    extensions.wrapfunction(merge, 'update', wrapupdate)
    extensions.wrapfunction(filemerge, '_xmerge', _xmerge)
```

We also have to pull all of the transitive dependencies for this logic
into `hgevents`.

Finally, we also have to define a `reposetup()` function in `hgevents`
that does a subset of what `reposetup()` does in `fsmonitor`. Specifically,
it ensures that a Watchman client is created for a `repo`, as appropriate,
so that it can be used to dispatch state changes to Watchman in
`state_filemerge` and `state_update`.

Note that the utility functions `createclientforrepo()` and
`getclientforrepo()` have been added to ensure that only one
Watchman client is created (and shared) when both `fsmonitor`
and `hgevents` are enabled.

Today, when an Hg repo is created in Eden, we set `extensions.fsmonitor=!`
in the `.hg/hgrc`:

diffusion/FBS/browse/master/fbcode/eden/hooks/hg/post-clone.py$69

Therefore, to get existing repos (both Eden and non-Eden) to pick up
the `hgevents` extension automatically, we add it to the list of
`[extensions]` in `common.rc`:

diffusion/FBS/browse/master/fbcode/scm/hg/fb/staticfiles/etc/mercurial/repo-specific/common.rc$53-60

as this is where `fsmonitor` is configured. We do not enable it in
`scm/hg/fb/staticfiles/etc/mercurial/facebook.rc` because
there is no reason to enable `hgevents` on Hg servers. Therefore, we
also decline to add `hgevents` to the set of `DEFAULT_EXTENSIONS` in
`scm/hg/mercurial/extensions.py`.

Reviewed By: quark-zju

Differential Revision: D8003628

fbshipit-source-id: 4f23881f8c25f4638f5475c292537b0352ae8d15
2018-05-21 09:30:28 -07:00
Michael Bolin
36eb4d26f8 mv fsmonitor/watchmanclient.py to extlib/watchmanclient/__init__.py
Summary:
There is some logic in `hgext/fsmonitor/watchmanclient.py` that needs
to be shared between fsmonitor and the new extension we are trying to
split out of it as part of T29379329.

Reviewed By: quark-zju

Differential Revision: D8003255

fbshipit-source-id: de01c5ba1460d7dde22969980b296afb3a942dd7
2018-05-15 12:06:47 -07:00
Michael Bolin
54b487b301 Move hgext/fsmonitor/pywatchman/ to hgext/extlib/pywatchman/
Summary:
This is a precursor to splitting the fsmonitor extension, as both
it and the new extension will use pywatchman.

Reviewed By: quark-zju

Differential Revision: D8002713

fbshipit-source-id: 37983fe2898d23223d1178eb3f15685f17ff8868
2018-05-15 12:06:47 -07:00
Durham Goode
18697e3fb3 hg: implement getmissing() for Rust hgstore
Summary:
Initial implementation of getmissing for a simple Rust pass through
data store. Future diffs will extend this to implement the union data store
completely in Rust.

Reviewed By: quark-zju

Differential Revision: D7632405

fbshipit-source-id: e660d33f8231410805cfaba6d77c56f27b002f8e
2018-05-14 12:05:13 -07:00
Durham Goode
f2b4d7f2e0 hg: implement getmeta() for Rust hgstore
Summary:
An initial implementation of getmeta for the Rust hg data store. Future
diffs will add more functionality.

Reviewed By: quark-zju

Differential Revision: D7632404

fbshipit-source-id: 53bd3b96b777bc3c5aef2b4d07ce1a9d9a5a52ed
2018-05-14 12:05:13 -07:00
Durham Goode
39dde8552d hg: implement getdeltachain() for Rust hgstore
Summary:
An initial implementation of getdeltachain for a simple pass through
data store. Future diffs will add additional functionality.

Reviewed By: quark-zju

Differential Revision: D7632407

fbshipit-source-id: 1a38089ba8ea70f8772af95afd871ee493082d80
2018-05-14 12:05:13 -07:00
Durham Goode
880ff5d0a9 hg: implement datastore.get() for Rust hgstore
Summary:
Implements the get function for a simple pass through rust data store
layer. Future diffs will implement more functions, and then later we will
implement the entire union data store in Rust.

Reviewed By: quark-zju

Differential Revision: D7632403

fbshipit-source-id: 3a1d0a8500e3110213d70dc1cff637cf8eadd809
2018-05-14 12:05:13 -07:00
Durham Goode
a97e97e413 hg: initial boiler plate for new hgstore crate
Summary:
This will contain all the Python centric hg store code that will let
Python call into the Rust storage layer.

Reviewed By: quark-zju

Differential Revision: D7632406

fbshipit-source-id: 6b7bcc8f47a23e9c0121e1f92de1137369bf584e
2018-05-14 12:05:12 -07:00
Jun Wu
d15213f6f5 treedirstate: move non-Python part to a separate crate
Summary:
This makes it easier to modify and test the core logic without coupling with
the Python logic.

Reviewed By: markbt

Differential Revision: D7734012

fbshipit-source-id: 0d7b19198d85f6ca7314611256e9271be60070d1
2018-04-24 15:59:07 -07:00
Jun Wu
58029547d6 cleanup: remove Cargo.lock from check-in
Summary:
They cause a lot of churn when updating vendored crates. It's unnecessary to
check them in because the vendored crates basically pinned versions of all
dependencies.

Also clean up .gitignore, since it does not support "syntax:re".

Reviewed By: DurhamG

Differential Revision: D7562533

fbshipit-source-id: b9fcade6f38c0652a1b19ac4f90827dac7eed181
2018-04-21 03:42:58 -07:00
David Lai
137fdc5b1c convert string compare to use equality operators
Summary:
The changes in this diff changes string comparisons using the compare method to
using equality operators.

Motivation:
- readability, simplifies code
- compare method is intended for sorting functions

This is clang check used: [Link Here](https://clang.llvm.org/extra/clang-tidy/checks/readability-string-compare.html)

Reviewed By: yfeldblum

Differential Revision: D7674416

fbshipit-source-id: dce375fb4b401f0e012ad4fc99b5dc4babe07f89
2018-04-19 13:54:59 -07:00
David Lai
a2411f090d - Fix unused parameter warnings
Summary:
The changes in this diff comments out unused parameters.
This will allow us to enable -Wunused-parameter as error.

Reviewed By: yfeldblum

Differential Revision: D7634823

fbshipit-source-id: fed215b805e9396d14feada19a6e959a180218d4
2018-04-16 10:35:14 -07:00
Mark Thomas
b6bdd61b72 treedirstate: sync all data and metadata when flushing
Summary:
When appending to the dirstate tree file, treedirstate flushes all data, and
then calls `sync_data` to ensure the data has made it to the disk.  This ought
to be sufficient, as the data won't be used until the dirstate file has been
updated to include the new root, which happens via atomic rename later on.

We've seen some cases where the dirstate file contains an invalid root id.
Attempt to mitigate this by syncing all data when we flush the dirstate.  There
is a performance penalty to this, but it shouldn't be too much.

Differential Revision: D7599547

fbshipit-source-id: 9d55b80d04833d2a73f058487a62eab2da802214
2018-04-13 21:51:54 -07:00
Kostia Balytskyi
64b26c3c94 hg: migrate cfastmanifest code from system to portability headers
Summary: To allow for Windows compilation.

Differential Revision: D7568270

fbshipit-source-id: eeef1b3712a30da3263ac2e2554d26f2aa48ff1a
2018-04-13 21:51:51 -07:00
Kostia Balytskyi
446d302f33 hg: do not use void* arithmetics not supported by the standard
Summary:
Yet another gcc-specific behavior that we're using.

FYI: https://stackoverflow.com/a/3524270/440677

Differential Revision: D7568272

fbshipit-source-id: ffa033ee9ec521ca0b4ac4befe447929c93db273
2018-04-13 21:51:51 -07:00
Kostia Balytskyi
dc1583e347 hg: remove extra semicolons in structs
Summary:
`PyObject_HEAD` is macro that ends with a semicolon when is is resolved. Thus `PyObject_HEAD;` ends in two semicolons.
Tolerating extra semicolons is a gcc feature, not a standard one.
I am adding the explanation for `// clang-format off` only once per file.

https://stackoverflow.com/questions/49529130/are-extra-semicolons-allowed-inside-struct-declarations?rq=1&utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa

Differential Revision: D7568269

fbshipit-source-id: 5afcfd7d96dc638cd0673d25809c9c43d58d00de
2018-04-13 21:51:51 -07:00
Kostia Balytskyi
3fd779e8be hg: migrate datapackstore to portability's dirent.h
Summary: Just start including a different header.

Differential Revision: D7555760

fbshipit-source-id: 7b0cd273a01db29cc31044b0cc0cff144b13a22b
2018-04-13 21:51:50 -07:00
Jun Wu
e83f8cfa4d rust: add more dependencies
Summary: Add crates that are used in the upcoming matcher implementation.

Differential Revision: D7319606

fbshipit-source-id: c269aa9afac635838940239f3e5ab173c38b2429
2018-04-13 21:51:31 -07:00
Kostia Balytskyi
0ef59877cd hg: some portability fixes to py-cdatapack.h
Summary:
1. Variable Length Arrays are not supported by MSVC, but since this is a C++ code, we can just use heap allocation
2. Replacing `inet` with portability version

Depends on D7196403

Reviewed By: quark-zju

Differential Revision: D7196605

fbshipit-source-id: a0d88b6e06f255ef648c0b35a99b42ba3bee538a
2018-04-13 21:51:24 -07:00
Mateusz Moneta
4cfb665650 Update markers during hg pullbackup
Summary:
Before this change `hg pullbackup` did not set correct markers on commits.

This change make possible to see what changes already landed even when we are restoring repository from backup.
Before the change `fbclone` + `hg pullbackup` of repo with `C1` commit landed would result in:
```
o  o C2
|    |
o  o C1
|  /
|
o
```
after:
```
o  o C2
|    |
o  x C1
|  /
|
o
```

Reviewed By: StanislavGlebik

Differential Revision: D7032572

fbshipit-source-id: ffee3c7cc23c24a3df9a89c999c9dd2de226dbff
2018-04-13 21:51:23 -07:00
Mark Thomas
87aa45f2ce vendorcrates: update to latest vendored crates
Summary: Update rust dependencies to the latest versions, including cpython 0.2.0.

Reviewed By: quark-zju

Differential Revision: D7101611

fbshipit-source-id: dc9cace3237b9a003766aa82f4abc6192e13fa06
2018-04-13 21:51:19 -07:00
Durham Goode
75da4fb2e6 hg: add add/removeStore to cuniondatapackstore
Summary:
In a future diff we'll need the ability to modify the union store on
the fly, so let's add addstore and removestore apis.

Reviewed By: ryanmce

Differential Revision: D7051102

fbshipit-source-id: 901a50720bfdf4e5c59714d092830e65edccdfce
2018-04-13 21:51:17 -07:00
Durham Goode
f0d99a2e09 hg: clang format the cstore code
Summary: The linters complain about this now, so let's format everything.

Reviewed By: ryanmce

Differential Revision: D7057989

fbshipit-source-id: 987ad0dcaa2f4e8fb74b3aa19c496f378765a533
2018-04-13 21:51:17 -07:00