Summary:
Implement autopull so non-automation `hg pull -r Dxxxx`, `hg up Dxxxx` will
pull `Dxxxx` automatically.
Since we now autopull the commits, error messages are removed. The old code
actually causes issues because it will raise at `"D1234" in repo`, which is
a surprise to many code paths, including the revset autopull logic, which
uses `x in repo` to decide whether `x` is unknown.
Note `hg pull -r Dxxxx` is not using the revset layer and needs to be handled
separately. It works for hg servers right now because the server can translate
`Dxxx` to a commit hash. It probably does not work for a Mononoke server.
Reviewed By: sfilipco
Differential Revision: D21320626
fbshipit-source-id: 939abe12e3a9a8ed5ca7ed29bb4f90fb39e7674a
Summary:
Change the interface to return infallible `Optional[node]` instead of fallible
`List[rev]`. In the next diff, we're going to use the 'node' information to
implement autopull.
Reviewed By: sfilipco
Differential Revision: D21320628
fbshipit-source-id: 6f7c070faba667cc85313cc78d6149c787ca8593
Summary:
Currently, phrevset picks the "successor" that has the maximum revision
number. That depends on the assumption that larger revision numbers
are modified last. Howevver, that's not always true with commit cloud sync. For example, in my repo I have D21179514 modified from 63768bf43
to 684612d5d. Phabricator has 63768bf43. Local successor is 684612d5d,
but the revision number of 684612d5d is smaller than 63768bf43:
quark@devvm1939 ~/hg/edenscm/hgext % hg log -r 'D21179514'
changeset: 63768bf436d01982a8d42ce97160ac6d9ae2cdad D21179514
user: Jun Wu <quark@fb.com>
date: Wed, 22 Apr 2020 09:45:50 -0700
summary: [hg] commitcloud: log metalog root during update references
quark@devvm1939 ~/hg/edenscm/hgext % lhg log -r 'D21179514'
changeset: 684612d5d606b01c224889f2b3f87aff7b93db49 D21179514 (@)
user: Jun Wu <quark@fb.com>
date: Wed, 22 Apr 2020 10:10:37 -0700
summary: [hg] commitcloud: log metalog root during update references
quark@devvm1939 ~/hg/edenscm/hgext % lhg log -r 'successors(63768bf436d0198
2a8d42ce97160ac6d9ae2cdad)' -T '{node} {rev}\n'
684612d5d606b01c224889f2b3f87aff7b93db49 76718
63768bf436d01982a8d42ce97160ac6d9ae2cdad 95363
Improve it by actually prefer selecting a non-obsoleted successor.
Reviewed By: sfilipco
Differential Revision: D21267552
fbshipit-source-id: d43d72a7c273c55af70bb41ad967fff0c78a452a
Summary: This is more efficient (if pullattempts have high chance to succeed).
Reviewed By: sfilipco
Differential Revision: D21320627
fbshipit-source-id: a2166f5a59f98c7d705c806b9d152ceb9981f3be
Summary:
Make it so that the autopull functions just describe what to do (using the
`pullattempt` struct) instead of having the side effect directly. This will be
useful in multiple cases:
- The actual autopull logic becomes easier to implement.
- The revset autopull layer can merge `pullattempt`s to fetch things in one go.
- The `pull -r` logic can reuse the `pullattempt`s to translate what to pull
(ex. from `D1234` to a commit hash).
This has subtle changes that multiple "remote"s are no longer properly
supported. That is probably fine in our production use-cases but we might
want to revisit if we want to support remotes "properly". Currently, this
greatly simplifies things due to the fact that "infinitepush" or
"infinitepushbookmark" have to be used in certain cases.
Reviewed By: sfilipco
Differential Revision: D21320633
fbshipit-source-id: e38b68abf69a34a97431685aa7ab0d2fe022fda8
Summary: Add a way for extensions to register how to auto pull unknown names.
Reviewed By: sfilipco
Differential Revision: D21320630
fbshipit-source-id: d20e11cff83b8a7e15a5085b0508a3e5bef305c3
Summary:
We're going to make autopull more complicated. Move it to a separate module as
it is not directly related to revset.
Reviewed By: sfilipco
Differential Revision: D21320631
fbshipit-source-id: fe60bc53ebf1c75f8bf66156805cbe2801fe6532
Summary:
The revset optimization makes unknown names harder to extract. For example,
`(or (list "a" "b" "c"))` will be optimized to `(_list "a\0b\0c")` and it
becomes harder to extract `"a"`, `"b"`, and `"c"`.
Move the unknown name extraction logic to before the optimization step to solve
it.
Reviewed By: sfilipco
Differential Revision: D21320632
fbshipit-source-id: 3a25f1cf4aab0449be6952113d622f29b1c0b631
Summary:
vs2017 is not able to compile the static assertion in KeySpace.cpp.
Previously we thought that this would be resolved in a later release of vs2017
but now that is here it is clear that it hasn't been fixed.
This commit pushes the version requirement to vs2019 (see
https://dev.to/yumetodo/list-of-mscver-and-mscfullver-8nd for a mapping between
product versions and compiler versions), but we cannot build with vs2019
because folly and rangev3 don't compile with vs2019, so this assertion (heh!)
has literally not been tested.
This commit also fixes up an oversight in the gating logic: the intent is that
we perform the assertion on all systems except known broken MSVC. We were
accidentally restricting it to later versions of MSVC.
Reviewed By: simpkins
Differential Revision: D21432890
fbshipit-source-id: e11ffccc53bf8dffdf2db45ad4f3cf199b1cc70d
Summary:
This allows us to understand what config is used during a transaction.
For example, is `selectivepull` enabled during a `pull`?
Reviewed By: DurhamG
Differential Revision: D21222146
fbshipit-source-id: a8c82f2b02e9657885947a706f728e28b1bfc1e2
Summary:
We're seeing deadlocks where if the process forks (like in a update
worker) while a background python thread is holding the progress lock, it can
cause a deadlock in the forked process if the thread ever tries to access the
progress bar.
To fix it, let's invalidate the engine if the process forks.
Reviewed By: xavierd
Differential Revision: D21415152
fbshipit-source-id: 75607dd2c1ed122b3a9df68d359ba9dcdde78a77
Summary:
D21364132 accidentally broke this; we can't run the fetcher
for projects for which we pulled the build out of cache, because there
is no source to update in that case.
This commit adjusts the logic so that we write out a marker file
to indicate that we installed a build from cache and to look for
that file being present to gate the new update logic.
Reviewed By: lnicco
Differential Revision: D21419122
fbshipit-source-id: 304670848add22531d88549d66f22c40ff255140
Summary:
There was a bug in scrub blobstore that caused failures while traversing the
fsnodes.
If all blobstores returned None, then we need to return None and not fail as we
do now. So the situation we ran into was:
1) fsnodes is not derived, all blobstore return None
2) Previously it returned the error, which later checked in
https://fburl.com/diffusion/mhhhnkxv - this check makes sure there's no entry
with the same key on the queue. However by that time fsnodes might already be
derived and someone else might insert a new entry in the blobstore and in the
queue. This would return an error to the client.
The fix here is to not fail if all blobstores returned None
Reviewed By: ahornby
Differential Revision: D21405418
fbshipit-source-id: 21fe130ce65a0087c408a5014e5b108c7ce8fe6c
Summary:
A number of repo names are used quite frequently. Let's use an enum to
prevent typos and make things cleaner.
Reviewed By: quark-zju
Differential Revision: D21365036
fbshipit-source-id: 1d3d681443df181e9076f5ee87029ae61124a486
Summary: Cover as much as remining code with `Cargo.toml`s, for the rest create an exlusion list in the autocargo config.
Reviewed By: krallin
Differential Revision: D21383620
fbshipit-source-id: 64cc78a38ce0ec482966f32a2963ab4939e20eba
Summary: Covering repo_listener and microwave plus some final touch and we have a buildable Mononoke binary.
Reviewed By: krallin
Differential Revision: D21379008
fbshipit-source-id: cca3fbb53b90ce6d2c3f3ced7717404d6b04dd51
Summary:
There are few related changes included in this diff:
- backsyncer is made public
- stubs for SessionContext::is_quicksand and scuba_ext::ScribeClientImplementation
- mononoke/hgproto is made buildable
Reviewed By: krallin
Differential Revision: D21330608
fbshipit-source-id: bf0a3c6f930cbbab28508e680a8ed7a0f10031e5
Summary:
- Change get return value for `Blobstore` from `BlobstoreBytes` to `BlobstoreGetData` which include `ctime` metadata
- Update the call sites and tests broken due to this change
- Change `ScrubHandler::on_repair` to accept metadata and log ctime
- `Fileblob` and `Manifoldblob` attach the ctime metadata
- Tests for fileblob in `mononoke:blobstore-test` and integration test `test-walker-scrub-blobstore.t`
- Make cachelib based caching use `BlobstoreGetData`
Reviewed By: ahornby
Differential Revision: D21094023
fbshipit-source-id: dc597e888eac2098c0e50d06e80ee180b4f3e069
Summary: Instead of having `HgBackingStore` fetch blobs separately, we now try to read from hgcache and fetch from `HgImporter` as batches.
Reviewed By: chadaustin
Differential Revision: D20903245
fbshipit-source-id: d8e404d6765f1bcbacbf2a39f83eab0a351a3fe0
Summary: This diff makes `HgImportRequestQueue` to be able to return multiple requests in the queue at once.
Reviewed By: chadaustin
Differential Revision: D20197070
fbshipit-source-id: 8cff1780d6e56321a756d30ac0e9b9d5d319c049
Summary: The life of a request is only able to be finished with one of the method. So we can instead having the tracker destroyed when the future is resolved.
Reviewed By: chadaustin
Differential Revision: D20995819
fbshipit-source-id: 5dac2f762513b5d0bcacaab7d0669fc8fdb61e80
Summary:
Readability improvements.
`HgQueuedBackingStore` will need to call these functions individually.
Reviewed By: chadaustin
Differential Revision: D20683321
fbshipit-source-id: 9a9bd766c34559048bd0971f17304090abbb2774
Summary:
During a graceful restart, while the fuse mount is not handling
requests, avoid a possible deadlock between the edenfs, hg
debugedenimporthelper, and watchman processes.
See the comment in HgImporter.cpp.
Reviewed By: fanzeyi
Differential Revision: D21342275
fbshipit-source-id: df8fb5df5d5cd1490e88b42054b34cbb2acdb692
Summary: This bug got in while iterating the original Diff. It should only be returning empty when the blob does not exist locally.
Reviewed By: xavierd
Differential Revision: D21417659
fbshipit-source-id: 676e22313ab4a024af5341d8c99797fc062bd293
Summary:
Instead of trying to maintain two hgrc.dynamic's for shared repositories,
let's just always use the one in the shared repo. In the long term we may be
able to get rid of the working-copy-specific hgrc entirely.
This does remove the ability to dynamically configure individual working copies.
That could be useful in cases where we have both eden and non-eden pointed at
the same repository, but I don't think we rely on this at the moment.
Reviewed By: quark-zju
Differential Revision: D21333564
fbshipit-source-id: c1fb86af183ec6dc5d973cf45d71419bda5514fb
Summary: Let's log the mismatches to scuba so we can track them down.
Reviewed By: quark-zju
Differential Revision: D21313255
fbshipit-source-id: ee79c9504a80ebe5b21849c0eae5993b65eaff28
Summary:
In a future diff we want to log information about configuration
mismatches. In order to do that, we need to be able to call ui.log before
extensions are setup. Let's move the ui.log sampling logic into core so it can
be called before extension setup.
Reviewed By: quark-zju
Differential Revision: D21313257
fbshipit-source-id: fe1c0f572720c17e7398f2a4fa7082ef8fb59536
