Summary:
Debug output belongs to stderr.
This makes it possible to turn on debug output without breaking programs
parsing stdout.
Reviewed By: singhsrb
Differential Revision: D25315954
fbshipit-source-id: c7813a824fbf6640cb5b80b5ed2d947e7059d53e
Summary:
With `collapse-obsolete`, `.` can be obsoleted and in the middle of a stack and
not shown up. That can be confusing. Make the smartlog revset always show the
`heads` passed in. If `.` is in `heads` (the default), then show it.
Reviewed By: DurhamG
Differential Revision: D24696595
fbshipit-source-id: 7deab109d0e0ae5e703928252bc63312d936955f
Summary:
Add open_existing_sqlite_path so we don't doing create_dir_all when we know the db already exists.
Noticed it in passing while investigating something else.
Reviewed By: markbt
Differential Revision: D25469502
fbshipit-source-id: 9810489c84220927937c037d69f5e8e70f2d9038
Summary: * suggest connecting to the VPN if hg update doesn't work
Reviewed By: sfilipco
Differential Revision: D25551017
fbshipit-source-id: 575f29cce4ab2719f2faae86616fdd9aac739f5f
Summary:
If Rust LFS is in use, we currently don't upload LFS blobs to commit cloud.
This is problematic because if you're going to Mononoke that means you can't
upload, and if you're going to Mercurial that means you're silently not backing
up data.
Reviewed By: StanislavGlebik
Differential Revision: D25537672
fbshipit-source-id: fd61f5a69450c97a0bc0895193f67fd22c9773fb
Summary: These show up when compiling with Buck, let's silence them.
Reviewed By: chadaustin
Differential Revision: D25513672
fbshipit-source-id: 277afae30059114f3646cdf4feedac442a4ee1b6
Summary:
On Windows, the GUID of the mount point identifies the virtualization instance,
that GUID is then propagated automatically to the created placeholders when
these are created as a response to a getPlaceholderInfo callback.
When the placeholders are created by EdenFS when invalidating directories we
have to pass GUID. The documentation isn't clear about whether that GUID needs
to be identical to the mount point GUID, but for a very long time these have
been mismatching due to the mount point GUID being generated at startup time
and not re-used.
One of the most common issue that users have reported is that sometimes
operations on the repository start failing with the error "The provider that
supports file system virtualization is temporarily unavailable". Looking at the
output of `fsutil reparsepoint query` for all the directories from the file
that triggers the error to the root of the repositories, shows that one of the
folder and its descendant don't share the same GUID, removing it solves the
issue.
It's not clear to me why this issue doesn't always reproduce when restarting
EdenFS, but a simple step that we can take to solve this is to always re-use
the GUID, and that hopefully will lead to the GUID always being the same and
the error to go away.
Reviewed By: fanzeyi
Differential Revision: D25513122
fbshipit-source-id: 0058dedbd7fd8ccae1c9527612ac220bc6775c69
Summary:
The config verifier would remove items from the values list if they
were disallowed. To do this, it iterated through the values list backwards,
removing bad items. In some cases it stored the index of a bad value for later
use, but because it was iterating backwards and removing things, the indexed it
stored might not be correct by the time the loop is done. To fix this, let's go
back to iterating forwards.
Reviewed By: quark-zju
Differential Revision: D25539737
fbshipit-source-id: 87663f3c162c690f3961b8075814f3467916cb4b
Summary:
The old case-conflict checks were more lenient, and only triggered if a commit
introduced a case conflict compared to its first parent.
This means that commits could still be landed to bookmarks that already had
pre-existing case conflicts.
Relax the new case-conflict checks to allow this same scenario.
Note that we're still a bit more strict: the previous checks ignored other
parents, and would not reject a commit if the act of merging introduces a case
conflict. The new case conflict checks only permit case conflicts in the case
where all conflicting files were present in one of the parents.
Reviewed By: StanislavGlebik
Differential Revision: D25508845
fbshipit-source-id: 95f4db1300ee73b8e6495ba8b5c1c2ce5a957d1a
Summary: Spotted this in passing. Was able to remove a call to fetch_root_manifest_id.
Reviewed By: StanislavGlebik
Differential Revision: D25472678
fbshipit-source-id: d450cb97630464be13d22fb37c3356611dc2e1b6
Summary: This makes it easier to run full walks on small repos.
Reviewed By: StanislavGlebik
Differential Revision: D25469485
fbshipit-source-id: 6e5b1426837a396d939e47a5b353e615437ae7cb
Summary:
If .buckd/pid is empty or invalid, assume that means Buck is not
running.
Reviewed By: genevievehelsel
Differential Revision: D25544725
fbshipit-source-id: 101ef67e17ff3e06f428cd7dbf51b2587fee4627
Summary:
Restore the behavior disabled by D25350916 (49c6f86325). This time it no longer runs Python
logic in background threads.
Reviewed By: sfilipco
Differential Revision: D25513054
fbshipit-source-id: 0220ccb37e658518d105bba04f45424c9fcfe142
Summary:
Make `get_commit_raw_text` aware of hg's hardcoded commit hashes: NULL_ID and
WDIR_ID. Previously, only `stream_commit_raw_text` is aware of it.
This makes it a bit more compatible when used in more places.
Reviewed By: sfilipco
Differential Revision: D25515006
fbshipit-source-id: 08708734a28f43acf662494df69694988a5b9ca0
Summary:
Unlike streamcommitrawtext, the new API does not put Python logic to a
background thread. This will make it easier to reason about Python logic as
they do not need to be thread-safe, and we don't need to think about Python GIL
deadlocks in the Rust async world.
Reviewed By: sfilipco
Differential Revision: D25513057
fbshipit-source-id: 4b30d7bab27070badd205ac1a9d54bae7f1f8cec
Summary:
Previously, only the batch fetching, or the stream fetching APIs will
actually fetch commit remotely. The 1-commit fetching API does not have
the network side effect, with the hope that we can migrate all usecases
to stream or batch fetching.
Practically it's quite difficult to migrate all use-cases, and the Python
layer has to have a fallback 1-by-1 fetching. Now let's just move that
fallback to Rust to simplify the code. The fallback in the Rust code
is by the default impl of get_commit_raw_text.
Reviewed By: sfilipco
Differential Revision: D25513056
fbshipit-source-id: b3c615397d33b8d35876dc23ca7b95173783ef80
Summary: The API will be used in Python bindings to avoid running Python in background threads.
Reviewed By: sfilipco
Differential Revision: D25513055
fbshipit-source-id: a108b55115271a256c0d43e0ff7b82c0b209be81
Summary:
Previously only `iterctx` does prefetch. Make `__iter__` do prefetch via `iterctx`.
The old `__iter__` without prefetching was renamed to `iterrev`.
Reviewed By: sfilipco
Differential Revision: D24365404
fbshipit-source-id: db5c687066794257719bb64c673dc384b5460ff1
Summary:
Now smartset has a reference to repo. It does not need `repo` from external
source.
Reviewed By: sfilipco
Differential Revision: D24365405
fbshipit-source-id: 8a43697b7b84a8a41691ed8f095c271107a90f16
Summary:
This will make `__iter__` to be able to do proper prefetch, or make it possible
for `__iter__` to return node instead of rev. To avoid cycles, weakref is used.
The smartset types are used widely. It's hard to migrate all callsites at once.
For now, `repo` is optional. Later, it will be required.
Reviewed By: DurhamG
Differential Revision: D24365400
fbshipit-source-id: 5dd40e3d930893c39f16da8f3169b026c8933bd2
Summary:
The test is one of the most fragile tests (ex. it has traceback that is
fragile). Since we don't support 3rd party extensions the test is less
meaningful. Let's remove it.
Reviewed By: singhsrb
Differential Revision: D25514727
fbshipit-source-id: cc3491597ba38ad84c4c2b69a275da6505ae2207
Summary:
It has a high chance of failing with:
```
$ hg --config extensions.b=b.py --config 'pager.pager=head -1' init foo2
+ close failed in file object destructor:
+ IOError: [Errno 9] Bad file descriptor
line1
```
when running with chg together with other tests.
Reviewed By: singhsrb
Differential Revision: D25514728
fbshipit-source-id: d9d2adea97bc6a436341be6f6d9aac43cb78c90a
Summary:
This is a bit tricky - look-ahead does not work because the content of commit
texts decide whether the commit is in the resulting set or not.
Add a special case for `ancestorsaged` so it does not end up with 1-by-1
fetching.
Reviewed By: sfilipco
Differential Revision: D24324797
fbshipit-source-id: 844ac6a6637e25900eea00d99704a59dfc7a0345
Summary:
Some revset function need commit text data to be efficient. Prefetch commit
text explicitly for them.
Reviewed By: DurhamG
Differential Revision: D24322227
fbshipit-source-id: 49c0b4f8cd9d9a744e10d169a1b1247faa8edd8e
Summary: Made unnecessary by using 2018 edition for rust_bindgen_library targets (D25441322).
Reviewed By: jsgf
Differential Revision: D25441329
fbshipit-source-id: d00aaad09451c77c6d05ed5d671468a481ce4e25
Summary:
Some of our tests are marked as flaky because of timeouts. They usually run
under timeout but because it's integration tests the times vary a lot.
The `#requre slow` is marking those tests as slow giving them more time to tun.
Reviewed By: krallin
Differential Revision: D25495555
fbshipit-source-id: 02bc3755992f56f5e743835318cf1233ab7c623d
Summary:
Now that the mapping is separated from BonsaiDerivable, it becomes clear where
batch derivation is incorrectly using the default mapping, rather than the
mapping that has been provided for batch-derivation.
This could mean, for example, if we are backfilling a v2 of these derived
data types, we could accidentally base them on a v1 parent that we obtained
from the default mapping.
Instead, ensure that we don't use `BonsaiDerived` and thus can't use the
default mapping.
Reviewed By: krallin
Differential Revision: D25371963
fbshipit-source-id: fb71e1f1c4bd7a112d3099e0e5c5c7111d457cd2
Summary:
The backfiller may read or write to the blobstore too quickly. Apply QPS
limits to the backfill batch context to keep the read or write rate acceptable.
Reviewed By: ahornby
Differential Revision: D25371966
fbshipit-source-id: 276bf2dd428f7f66f7472aabd9e943eec5733afe
Summary:
The common case of limiting blobstore rates using a leaky bucket rate limiter
is cumbersone to set up. Create a convenience method to do it more easily.
Reviewed By: ahornby
Differential Revision: D25438685
fbshipit-source-id: 821eda7bd0ddf71f22378c1b23e66b6d3f6454e7
Summary:
When fetching many derived data mappings, the use of `FuturesUnordered` means
we may fetch many blobs concurrently, which may overload the blobstore.
Switch to using `buffered` to reduce the number of concurrent blob fetches.
Reviewed By: ahornby
Differential Revision: D25371965
fbshipit-source-id: 30417e86bc33defbb821f214a5520ab1b8a8c18c
Summary:
Large batches with parallel derivation can cause problems in large repos.
Allow control of the batch size so that it can be reduced if needed.
Reviewed By: krallin
Differential Revision: D25401205
fbshipit-source-id: 88a76a7745c34e4e34bc9b3ea9228bd5dad857f6
Summary:
Re-introduce parallel backfilling of changesets in a batch using `batch_derive`,
however keep it under the control of a flag, so we can enable or disable it as
necessary.
Reviewed By: ahornby
Differential Revision: D25401207
fbshipit-source-id: f9aeef3415be48fc03220c18fa547e05538ed479
Summary:
Change derived data config to have "enabled" config and "backfilling" config.
The `Mapping` object has the responsibility of encapsulating the configuration options
for the derived data type. Since it is only possible to obtain a `Mapping` from
appropriate configuration, ownership of a `Mapping` means derivation is permitted,
and so the `DeriveMode` enum is removed.
Most callers will use `BonsaiDerived::derive`, or a default `derived_data_utils` implementation
that requires the derived data to be enabled and configured on the repo.
Backfillers can additionally use `derived_data_utils_for_backfill` which will use the
`backfilling` configuration in preference to the default configuration.
Reviewed By: ahornby
Differential Revision: D25246317
fbshipit-source-id: 352fe6509572409bc3338dd43d157f34c73b9eac
Summary:
Currently, data derivation for types that have options (currrently unode
version and blame filesize limit) take the value of the option from the
repository configuration.
This is a side-effect, and means it's not possible to have data derivation
types with different configs active in the same repository (e.g. to
server unodes v1 while backfilling unodes v2). To have data derivation
with different options, e.g. in tests, we must use `repo.dangerous_override`.
The first step to resolve this is to make the data derivation options a parameter.
Depending on the type of derived data, these options are passed into
`derive_from_parents` so that the right kind of derivation can happen.
The mapping is responsible for storing the options and providing it at the time
of derivation. In this diff it just gets it from the repository config, the same
as was done previously. In a future diff we will change this so that there
can be multiple configurations.
Reviewed By: krallin
Differential Revision: D25371967
fbshipit-source-id: 1cf4c06a4598fccbfa93367fc1f1c2fa00fd8235
Summary: Take the parameters to `derived_data_utils` and `derived_data_utils_unsafe` by reference.
Reviewed By: krallin
Differential Revision: D25371970
fbshipit-source-id: d260650c2398e33667e1bc5779fbabdff04f1f98
Summary:
The `BonsaiDerived` trait is split in two:
* The new `BonsaiDerivable` trait encapsulates the process of deriving the data, either
a single item from its parents, or a batch.
* The `BonsaiDerived` trait is used only as an entry point for deriving with the default
mapping and config.
This split will allow us to use `BonsaiDerivable` in batch backfilling with non-default
config, for example when backfilling a new version of a derived data type.
Reviewed By: krallin
Differential Revision: D25371964
fbshipit-source-id: 5874836bc06c18db306ada947a690658bf89723c
Summary: Looks like the permissions are different there. Let's glob it out.
Reviewed By: singhsrb
Differential Revision: D25507359
fbshipit-source-id: 6a5c19e41879798b829d9b6e79eba3009249c20c
Summary:
At the moment "hg pull -B bookmark" always fetches from infinitepush path even
if we do something like "hg pull -B master".
Let's fetch from infinitepush only if a bookmark matches scratch mather
Reviewed By: markbt
Differential Revision: D25460577
fbshipit-source-id: 6563dcd3423c6a7a70ea1c1f7acdaf5db5e21875
Summary: Two doctor tests don't pass on macOS, so disable them for now.
Reviewed By: genevievehelsel
Differential Revision: D25509814
fbshipit-source-id: c3fa92daefd4fda67335bdc66f56e35e94ae4e6a
Summary: Make the auth crate validate the user's certificate before returning it. This way we can catch invalid certs before trying to use them.
Reviewed By: sfilipco
Differential Revision: D25454687
fbshipit-source-id: ad253fb433310570c20f33dbd0d0bf11df21e966
Summary: Add a new module that can parse X.509 certificates and detect common issues (e.g., the certificate is missing, corrupt, or expired). This should allow us to provide better UX around certificate errors.
Reviewed By: sfilipco
Differential Revision: D25440548
fbshipit-source-id: b7785fd17fa85f812fd38de09e79420f4e256065
