Commit Graph

59941 Commits

Author SHA1 Message Date
Durham Goode
6b0014490c configs: implement dynamic and repo config loading in Rust
Summary:
In a future diff we'll enable dynamic and repo config loading purely
from Rust. To do so we need load functions for both cases.  A future diff will
call these.

The dynamicconfig loading is based off the Python equivalent in uiconfig.py

Reviewed By: quark-zju

Differential Revision: D22712624

fbshipit-source-id: ff46f6315fb80d4cd9e31d875ac60264563b12f2
2020-08-16 16:56:00 -07:00
Durham Goode
194e815245 configs: move HGRCPATH loading to load_system
Summary:
Previously load_system would skip loading if HGRCPATH was present and
then load_user would actually load the HGRCPATH. In an upcoming diff I add
load_dynamic, which happens after system but before user. The tests for
dynamicconfig depend on HGRCPATH being loaded when load_dynamic runs, so let's
move HGRCPATH loading up to load_system.

Reviewed By: quark-zju

Differential Revision: D22712627

fbshipit-source-id: 91175d9d7f85b9392ffea4af815a4facebbfe7c1
2020-08-16 16:56:00 -07:00
Durham Goode
ef9ba19dc5 configs: make Options clonable
Summary:
In a future diff we'll allow an outside caller to pass an Options down
to configparsers::hg::load() so that filters can be applied during loading. Inside
hg::load() we need to use the options multiple times with different values, so
let's make Options clonable.

Reviewed By: quark-zju

Differential Revision: D22712626

fbshipit-source-id: 975145f38d35afe7d4a6c8e87071b0fb0ae74797
2020-08-16 16:55:59 -07:00
Durham Goode
0cea385252 configs: remove config from repo.rs API
Summary:
A future diff will move all dynamic and repo config loading to be in
configparser. As part of this, let's simplify the repo.rs API to not pass
configs around everywhere.

Reviewed By: quark-zju

Differential Revision: D22712628

fbshipit-source-id: 79f23991aa826ce8b4f7430b45d7702efdc6b982
2020-08-16 16:55:59 -07:00
Durham Goode
26564596a1 utils: add background process utility
Summary:
Similar to the Python runbgcommand (extutil.py), this is a Rust utility that runs a
detached background process in a cross platform way.

This will be used in a later diff to run dynamicconfig generation in the
background.

Reviewed By: quark-zju

Differential Revision: D22712629

fbshipit-source-id: a317465bf03c96d977a203678e2bef13ce57cc12
2020-08-16 16:55:59 -07:00
Durham Goode
0b123ba41d configs: move Rust dynamicconfig generation into configparser::hg
Summary:
As part of moving all hg config loading and generation logic into Rust,
let's move the config generation logic from hgcommands and pyconfigparser to
configparser, unifying them at the same time.

Future diffs will move config loading in as well.

Reviewed By: quark-zju

Differential Revision: D22590208

fbshipit-source-id: d1760c404a6a5c57347df30713c20de55cfdb9a4
2020-08-16 16:55:59 -07:00
Durham Goode
7ff28d3e1c configs: move dynamicconfig into configparser
Summary:
A future diff will unify all config loading into configparser::hg, but
to do so we need dynamicconfig to live in configparser, so it can load
dynamicconfigs. Let's move everything in.

Reviewed By: quark-zju

Differential Revision: D22587237

fbshipit-source-id: 5613094175b6e1597aa113ee3e6d92ce7ec79f6d
2020-08-16 16:55:59 -07:00
Durham Goode
a40331be8d configs: unify system+user config loading into pure rust layer
Summary:
We had two spots that loaded system and user configs, one in the
pyconfigparser layer, and one in the pure rust config layer. In an upcoming diff
I'd like to move dynamicconfig loading down into the pure rust layer, so let's
unify these.

Reviewed By: quark-zju

Differential Revision: D22585554

fbshipit-source-id: 0cea7801ae1d5a3a3c12b80ee23b37f9e690e2bc
2020-08-16 16:55:59 -07:00
Durham Goode
3129f032a4 contentstore: make history rotatelog size configurable
Summary:
In a future diff we'll increase the size of the rotatelog temporarily
during clones. To do so we need it to be configurable.

Reviewed By: quark-zju

Differential Revision: D23089539

fbshipit-source-id: ebfc3beaf3c0fe5b01b87d97c19455b0a24afa72
2020-08-16 16:44:16 -07:00
Durham Goode
b821ab3766 contentstore: make data rotatelog size configurable
Summary:
In a future diff we'll increase the size of the rotatelog temporarily
during clones. To do so we need it to be configurable.

Reviewed By: quark-zju

Differential Revision: D23089541

fbshipit-source-id: 5010e417a83a2611283322f1dbb7023f4286f503
2020-08-16 16:44:16 -07:00
Durham Goode
76d3d46837 revisionstore: remove from_path from LocalStore
Summary:
from_path is an awkward constructor because it doesn't pass any other
information, like a config object. It also requires that the constructor be very
generic across all the stores. Right now it's only needed for pack files, so
let's move it to it's own trait that is limited to pack files.

This will allow us to make the indexedlog store constructors more versatile in a
later diff. Once we get rid of pack files we can delete the StoreFromPath trait
entirely.

Reviewed By: xavierd

Differential Revision: D23089542

fbshipit-source-id: ea6c50853e5d5390a029002ef5d15c74fe41fe69
2020-08-16 16:44:16 -07:00
svcscm
5dc7a2a7e8 Updating submodules
Summary:
GitHub commits:

013b287cd9
7d1e1f8b53

Reviewed By: yns88

fbshipit-source-id: ba621446ab7f0d12d933f40897dd7fd35232beb2
2020-08-16 13:22:12 -07:00
svcscm
e418d3ad19 Updating submodules
Summary:
GitHub commits:

cf43d53b74

Reviewed By: yns88

fbshipit-source-id: 266cb07d4b6af56db60dc8dfe1b7015a4c316c7f
2020-08-15 13:11:36 -07:00
svcscm
195e0ed0a5 Updating submodules
Summary:
GitHub commits:

40df0f8a64

Reviewed By: yns88

fbshipit-source-id: 405213b1db6e7cd56d0f7dbdccead9eaa7d19490
2020-08-15 13:11:36 -07:00
svcscm
6df02c3aa6 Updating submodules
Summary:
GitHub commits:

03e2b8b90e

Reviewed By: yns88

fbshipit-source-id: 9de29685d96ec96fbdf7eb23a2789968ffe49dc3
2020-08-15 03:55:48 -07:00
Jun Wu
f61aaf3a15 repo: add dageval API
Summary:
Similar to `changelog.dageval`, but provides extra functions like `public`,
`draft`, `obsolete`, etc.

Reviewed By: sfilipco

Differential Revision: D23036070

fbshipit-source-id: b985f2b338a3dce11bddf53c00c30e4887762676
2020-08-14 22:00:26 -07:00
Jun Wu
a3531f9a8e changelog2: improve dageval
Summary:
Make it work with closure and `lambda dag: dag.only(...)`.
This is useful when there is a name conflict with a local variable, like:

   only = ...
   cl.dageval(lambda: only(...)) # only refers to the local variable.
   cl.dageval(lambda dag: dag.only(...)) # only refers to the dag operation.

Besides, drop `func_` so they are Py3 compatible.

Reviewed By: sfilipco

Differential Revision: D23036064

fbshipit-source-id: 8d32a34c51b6ba945cda5be313f9d71464f813b7
2020-08-14 22:00:26 -07:00
Jun Wu
2db783bed8 revlogindex: make parent_revs fallible
Summary: If parent_revs gets an out-of-bound rev, it should fail.

Reviewed By: sfilipco

Differential Revision: D23036071

fbshipit-source-id: 7fae0fd5adf07ac3c933a29d7d06289d8d740c60
2020-08-14 22:00:26 -07:00
Jun Wu
0f838d7abf revlogindex: fix \0 header handling
Summary:
If the text starts with `\0`, the `\0` should be considered as part of the
uncompressed text instead of a separated header.

Reviewed By: sfilipco

Differential Revision: D22970575

fbshipit-source-id: 49e8a1a1ea42a3c4cf153b70f59fd0558dcfcede
2020-08-14 22:00:26 -07:00
Jun Wu
54a1a620d0 revlogindex: fix parent handling
Summary:
The parent handling is unsound when there are revs that are skipped. Fix it by
reasoning about commit hashes for parents.

Reviewed By: sfilipco

Differential Revision: D23036078

fbshipit-source-id: 8f710171471025cd48b3bd8f6ea57c68330eb8b8
2020-08-14 22:00:26 -07:00
svcscm
6e8017fde9 Updating submodules
Summary:
GitHub commits:

bc59f8b96f
69760b4d05

Reviewed By: yns88

fbshipit-source-id: b0bfc8b29cecbaa946a5cb16029a1ee6538ee1cd
2020-08-14 22:00:25 -07:00
svcscm
ca69914ed7 Updating submodules
Summary:
GitHub commits:

0e46975f32
e93f14fb7f
dc5a4252e1
209bde495e
01f4612acd

Reviewed By: yns88

fbshipit-source-id: ac83f388b01dc9745600c8ea3a36e1eb4380c189
2020-08-14 18:59:36 -07:00
Xavier Deguillard
f4f159537f utils: add a platform independent FileUtils
Summary:
Up to now, Windows had to have its own version of folly::{readFile, writeFile,
writeFileAtomic} as these only operate on `char *` path, which can only
represent ascii paths on Windows. Since the Windows version is slightly
different from folly, this forced the code to either ifdef _WIN32, or use the
folly version pretending that it would be OK. The Windows version was also
behaving slightly differently from folly. For instance, where folly would
return a boolean to indicate success, on Windows we would throw an exception.

To simplify our code, add type safety and unify both, we can implement our own
wrappers on top of either folly or Windows APIs.

We still have some code that uses folly::readFile but these should only be
dealing with filedescriptors. As a following step, we may want to have our own
File class that wraps a file descriptor/HANDLE so we can completely remove all
uses of folly::readFile.

Reviewed By: wez

Differential Revision: D23037325

fbshipit-source-id: 2b9a026f3ee6220ef55097abe649b23e38d9fe91
2020-08-14 18:56:33 -07:00
Arun Kulshreshtha
523013c808 cmdlib: remove extra comment slashes
Reviewed By: quark-zju

Differential Revision: D23111025

fbshipit-source-id: b8606c322439c41097d739df59b551a8432e7fe4
2020-08-14 18:04:26 -07:00
Xavier Deguillard
17776fbbc1 win: various micro-optimization for opendir/readdir
Summary:
After enabling strace profiling for the dispatcher, it became very clear that
once files are populated on disk, most of the notification in edenfs come from
listing directories. While looking at the code to find any improvements, I
picked up a couple of low hanging fruits and cleanup that should help (not
measured).

Reviewed By: chadaustin

Differential Revision: D23121434

fbshipit-source-id: 8a8b369f3be6ffb6097e6e12ab0f07af0a6d56b7
2020-08-14 17:35:50 -07:00
Xavier Deguillard
76ea3054de win: conditionally enable negative path caching
Summary:
From a quick experiment, this greatly cuts down on the amount requests to
nonexistent files. For instance, the .hg directory in folders is now only
looked up once and no longer afterwards.

Reviewed By: wez

Differential Revision: D23112343

fbshipit-source-id: 223134ca591054ae9ac2e839033bbd1b714443da
2020-08-14 17:35:50 -07:00
Xavier Deguillard
3fb1b56ea7 win: remove PrjfsChannel::removeCachedFile
Summary: This function was unused, remove it.

Reviewed By: wez

Differential Revision: D23112344

fbshipit-source-id: e37dd1e07067cf457eeb5d6c0af35db3dfab9b43
2020-08-14 17:35:50 -07:00
Xavier Deguillard
79c8f0e86e win: unify the EdenDispatcher lifetime management
Summary:
This is purely moving code around to be more similar between Unices/Windows.
At some point in the future, I'd like to see just one EdenDispatcher class in
the shared code with as few platform specific code (ie: ifdef) as possible,
this is small step in this direction.

Reviewed By: wez

Differential Revision: D23112345

fbshipit-source-id: feb276040aea106d5951af30c5d445b4cb654e01
2020-08-14 17:35:50 -07:00
svcscm
8698555a5e Updating submodules
Summary:
GitHub commits:

bc74b01fab
0ed58531b6
29c3fb1033
dee7e0c95d
510c66fb75
a6cbfa8ab1
ea90de3029
d3ffaba7d1

Reviewed By: yns88

fbshipit-source-id: ceb44bca6450ef9ff633dd38fa50d9be0b970ae2
2020-08-14 16:40:34 -07:00
Xavier Deguillard
f1da417cea win: use the strace logger
Summary:
On Linux, all the notifications are sent to the strace logger, let's do the
same on Windows. Whenever possible, I've tried to use the name of the Linux
syscalls that correspond to the notification.

Reviewed By: chadaustin

Differential Revision: D23105536

fbshipit-source-id: 31c6e545cdec93b0f751682e34c34b894b2890c4
2020-08-14 16:01:27 -07:00
Xavier Deguillard
e0ec7d8896 fsinfo: fix cargo test
Summary:
Somehow `make local` doesn't complain about missing features, but `cargo test`
does.

Reviewed By: singhsrb

Differential Revision: D23132496

fbshipit-source-id: cdbfe1faf194d61d86493a760e45fd38087d2956
2020-08-14 13:38:41 -07:00
Skotch Vail
0db33b30c5 sparse: debugsparsematch accepts filesets
Summary: Use filesets so that we don't have to worry about exceeding command line

Reviewed By: quark-zju

Differential Revision: D23008329

fbshipit-source-id: bbfef9ecf23b810f4fa6ca8a6de1bbeb9330ae09
2020-08-14 09:46:32 -07:00
Skotch Vail
8780a3f66b sparse: debugsparsematch temp, multiple, exclude sparse profiles
Summary: This diff has a unit test to make sure on the fly sparse profiles work. We also allow for more than one sparse profile, and for multiple exclusion sparse profiles.

Reviewed By: quark-zju

Differential Revision: D23004062

fbshipit-source-id: 15f00401084c89a77d8e7bdd8c7eb688e20d131b
2020-08-14 09:46:32 -07:00
svcscm
5768e63bee Updating submodules
Summary:
GitHub commits:

095cea22c7
7cdef5be5b
dd476d456a

Reviewed By: yns88

fbshipit-source-id: 553bc0da5b1639b80f79cf381d007a23f62c7644
2020-08-14 09:46:32 -07:00
svcscm
844ad8356d Updating submodules
Summary:
GitHub commits:

df78f1c130
32c32c3a59
599ce398e0
988a7b19d1

Reviewed By: yns88

fbshipit-source-id: 076f50ddd362a11e536463b26a768da2d6229dd7
2020-08-14 03:32:37 -07:00
Tao Chen
f55d4fc261 Remove old getSelection APIs from ServiceSelectorCache
Summary: Remove old getSelection APIs from ServiceSelectorCache

Reviewed By: ovoietsa

Differential Revision: D23070503

fbshipit-source-id: ea013e43215b8bc7aa463efe7b374d67749dda97
2020-08-14 03:21:12 -07:00
svcscm
bea7694e40 Updating submodules
Summary:
GitHub commits:

ab1788308a
e52041d316
70d9cbe57f
d63ed44218

Reviewed By: yns88

fbshipit-source-id: 28dc64a3a278339fb23a2348913575c4fe96af96
2020-08-14 02:56:41 -07:00
Alex Hornby
213edc10ce mononoke: limit queue peeking from scrub blobstore when one store has missing value
Summary: When running manual scrub for a large repo with one empty store, we are doing one peek per key.  For keys that have existed for some time this is unnecessary as we know the key should exist and slows down the scrub.

Reviewed By: farnz

Differential Revision: D23054582

fbshipit-source-id: d2222350157ca37aa31b7792214af4446129c692
2020-08-14 02:37:45 -07:00
Mark Thomas
dba3ef35ca repo_client: extract common calls to bookmarks_movement
Summary:
Extract the calls to bookmarks_movement to separate functions to avoid duplication and
make the post-resolve action functions easier to read.

Reviewed By: StanislavGlebik

Differential Revision: D23057045

fbshipit-source-id: c6b5a8cdb2399e89c174c3df844529d4b5309edf
2020-08-14 02:28:56 -07:00
Mark Thomas
c529e6a527 bookmarks_movement: refactor bookmark movement for pushrebase
Summary: Refactor control of movement of non-scratch bookmarks through pushrebase.

Reviewed By: krallin

Differential Revision: D22920694

fbshipit-source-id: 347777045b4995b69973118781511686cf34bdba
2020-08-14 02:28:55 -07:00
Mark Thomas
a16b88d1c5 pushrebase: remove OntoBookmarkParams and clean up interface
Summary:
Some parts of the `pushrebase` public interface will be re-exported from `bookmarks_movement`.

Clean these up in preparation:

* Remove `OntoBookmarkParams` as it is now a simple wrapper around `BookmarkName` that
  prevents us from using a reference.

* Make the bundle replay data `Option<&T>` rather than `&Option<T>`, allowing us to
  use the former when available.  The latter can be readily converted with `.as_ref()`.

* Rename `SuccessResult` to `Outcome` and `ErrorKind` to `InternalError`.

Reviewed By: krallin

Differential Revision: D23055580

fbshipit-source-id: 1208a934f979a9d5eb73310fb8711b1291393ecf
2020-08-14 02:28:55 -07:00
Mark Thomas
c59c2979d2 bookmarks_movement: refactor bookmark movement for force-pushrebase
Summary:
Refactor control of movement of non-scratch bookmarks through force-pushrebase
or bookmark-only pushrebase.  These are equivalent to ordinary pushes, and so
can use the same code path for moving the bookmarks.

This has the side-effect of enabling some patterns that were previously not
possible, like populating git mappings with a force-pushrebase.

Reviewed By: ikostia

Differential Revision: D22844828

fbshipit-source-id: 4ef71fa4cef69cc2f1d124837631e8304644ca06
2020-08-14 02:28:54 -07:00
Mark Thomas
279c3dcd8f bookmarks_movement: refactor bookmark movement for push
Summary: Refactor control of movement of non-scratch bookmarks through plain pushes.

Reviewed By: krallin

Differential Revision: D22844829

fbshipit-source-id: 2f1a89e1d0f69880f74b7bc135144bfb305a918e
2020-08-14 02:28:54 -07:00
Mark Thomas
e0bdff5188 bookmarks_movement: refactor scratch bookmark movement
Summary:
Refactor control of movement of scratch bookmarks to a new `bookmark_movement` crate
that will contain all bookmark movement controls.

Reviewed By: krallin

Differential Revision: D22844830

fbshipit-source-id: 56d25ad45a9328eaa079c13466b4b802f033d1dd
2020-08-14 02:28:53 -07:00
Alex Hornby
61046e3adb rust: point interment crate to internment master
Summary: Update internment to point at its latest master branch commit.  Upstream has merged my PR to use DashMap inside internment, but they haven't cut a new crates release yet.

Reviewed By: jsgf, krallin

Differential Revision: D23075070

fbshipit-source-id: 8f4ec0e3ddbefd672c3040fb174d1cf5f6c1a94a
2020-08-14 02:15:24 -07:00
Alex Hornby
d59dd787c5 mononoke: make blobstore ctime a bit easier to use
Summary: Ctime is an Option<i64>, so rather than as_ctime()/into_ctime() use the fact that it's fairly small and Copy to just .ctime()

Reviewed By: krallin

Differential Revision: D23081739

fbshipit-source-id: be62912eca02e5c29d7473d6f386d98df11000dd
2020-08-14 02:09:46 -07:00
Durham Goode
661d02d6d5 http_client: disable ssl revocation checking on Windows
Summary:
Windows defaults to checking a revocation server for ssl certs. Inside
our datacenter it can't reach the server and fails. We don't have this on for
any other platforms, so let's disable it.

Reviewed By: sfilipco

Differential Revision: D23121739

fbshipit-source-id: 4d44d2a065bf340a8f74332553deb09a9c61be9b
2020-08-13 23:17:28 -07:00
Ailin Zhang
58561f9df2 use Eden native import in prefetchBlobs
Summary: Previously we use HgImporter prefetch request in `prefetchBlobs()`, but using `getBlob()` can give us more control over the prefetch process later. So now `getBlob()` is used in `prefetchBlobs()` when `useEdenNativePrefetch` is configured as true.

Reviewed By: kmancini

Differential Revision: D22984848

fbshipit-source-id: 0bd0b1c5b50bb16da36f188915904d0223827dc3
2020-08-13 21:09:09 -07:00
svcscm
fdd4fe4a57 Updating submodules
Summary:
GitHub commits:

5cb7fbfb8a
08caf348fd

Reviewed By: yns88

fbshipit-source-id: 8f0b3ae17c8832cac69baa6d600ed080bd9b1980
2020-08-13 21:09:09 -07:00
svcscm
c672b224bc Updating submodules
Summary:
GitHub commits:

b72912a011
8cde5e001a
fbee1057be

Reviewed By: yns88

fbshipit-source-id: b77b6d76aa6e5e2f29c851c4ea5fa7782fb2c025
2020-08-13 12:40:57 -07:00