Summary:
Remove HgIdDataStore::get_delta and all implementations. Remove HgIdDataStore::get_delta_chain from trait, remove all unnecessary implentations, remove all implementations from public Rust API. Leave Python API and introduce "delta-wrapping".
MutableDataPack::get_delta_chain must remain in some form, as it necessary to implement get using a sequence of Deltas. It has been moved to a private inherent impl.
DataPack::get_delta_chain must remain in some form for the same reasons, and in fact both implenetations can probably be merged, but it is also used in repack.rs for the free function repack_datapack. There are a few ways to address this without making DataPack::get_delta_chain part of the public API. I've currently chosen to make the method pub(crate), ie visible only within the revisionstore crate. Alternatively, we could move the repack_datapack function to a method on DataPack, or use a trait in a private module, or some other technique to restrict visibility to only where necessary.
UnionDataStore::get has been modified to call get on it's sub-stores and return the first which matches the given key.
MultiplexDeltaStore has been modified to implement get similarly to UnionDataStore.
Reviewed By: xavierd
Differential Revision: D21356420
fbshipit-source-id: d04e18a0781374a138395d1c21c3687897223d15
Summary:
These 3 tests compile without issues on Windows. The RocksDB one is weird,
while it compiles with no hickups, I simply cannot run the resulting test
binary, and I'm not sure how to debug this. The local store one fails in folly.
Reviewed By: chadaustin
Differential Revision: D21393724
fbshipit-source-id: db90bf20a9d116bc8aa493703997c5e8da76eb1f
Summary: All the tests are passing.
Reviewed By: chadaustin
Differential Revision: D21341730
fbshipit-source-id: 90a3872b190879ec163935ff53703157028f87bc
Summary:
The modeFromEntryType and treeEntryTypeFromMode tests for symlinks and
executable had to be disabled as these function explicitely do not support
these. Since mode bits are a bit meaningless on Windows, this is probably OK.
Reviewed By: chadaustin
Differential Revision: D21341728
fbshipit-source-id: 86acf24d9ab67a02ecab33b7ebe82a456295fc3c
Summary: All of these tests are passing on Windows with no changes.
Reviewed By: chadaustin
Differential Revision: D21341729
fbshipit-source-id: 2b4d52751e74fa953bfe5143dc0c5735de2d34cf
Summary:
fboss-oss build links to hash that corresponds to tag v4.4.0 released on Jan 11 2016
```
repo_url = https://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git
rev = 92a0236a3cdf3438000834121b7ea8a09f1f52b1
```
The change is to update the iproute2 version to ```v4.12.0``` (July 5 2017) to match with the version used internally to Facebook
Reviewed By: shri-khare
Differential Revision: D21411714
fbshipit-source-id: fac606396e284193bb7199cf60d2601594bfa5f0
Summary:
All of these were simply NOT_IMPLEMENTED on Windows, but the code compiles
and doesn't break any existing tests. The underlying called functions might
have been implemented already, or are NOT_IMPLEMENTED, either way, this reduces
the amount of `#ifdef _WIN32`.
Reviewed By: chadaustin
Differential Revision: D21405622
fbshipit-source-id: bdc2de41d6a57e1c0b532e76eeb2c0c86180d558
Summary: What it says in the title. I'd like to set up alarms on this.
Reviewed By: farnz
Differential Revision: D21450584
fbshipit-source-id: 539299407cea84c67ff14b30184e8df4282415f8
Summary:
Update `contrib/check-code.py` to Python 3.
Mostly it was already compatible, however stricter regular expression parsing
revealed a case where one of our tests wasn't working, and as a result lots of
instances of `open(file).read()` existed that this test should have caught.
I have fixed up most of the instances in the code, although there are many
in the test suite that I have ignored for now.
Reviewed By: quark-zju
Differential Revision: D21427212
fbshipit-source-id: 7461a7c391e0ade947f779a2b476ca937fd24a8d
Summary: Reformat using a newer version of Black.
Reviewed By: quark-zju
Differential Revision: D21426337
fbshipit-source-id: 1ac7f6e85a06feec0d41e9509eca09194f421a1d
Summary:
The latest version of Black removes unneccessary parenthesis. Mercurial's
test-check-code currently uses extra parentheses to signal untranslated strings, so
Black's reformatting breaks this test.
Capitulate to Black by adding a new `_x` translation marker that means "untranslated".
Reviewed By: quark-zju
Differential Revision: D21426335
fbshipit-source-id: a6c26d7c6365c49530a7dee3a5f9ed71ff166835
Summary:
If a bundle comes from the commit cloud forward filler, we need to ignore
and not record it.
To do so, we need to start paying attention to stream-level params for the
first time.
Reviewed By: krallin
Differential Revision: D21427620
fbshipit-source-id: 9ee417cd2a5f2f5bb6ec342cd63071c6ca822475
Summary:
We want to be able to record all the bundles Mononoke processes to be later
replayed by Mercurail.
Reviewed By: krallin
Differential Revision: D21427622
fbshipit-source-id: b88e10e03d07dae35369286fe31022f36a1ee5cf
Summary: To make it easier to navigate the codebase the oss-only code will be from now on stored in a separate module, similarly to how the fbcode-only code is stored.
Reviewed By: markbt
Differential Revision: D21429060
fbshipit-source-id: aa7e80961de2897dae31bd0ec83488c683633b7a
Summary:
Something was up yesterday with the warm bookmarks cache. It started failing on
some hosts, and went out of sync for > 1 hour on some hosts. The logs reported
a lot of failures, but without any context they weren't super useful:
P130438422.
This adds a bit more logging. If this happens again, we'll be able to better
understand what happened.
Reviewed By: StanislavGlebik
Differential Revision: D21447043
fbshipit-source-id: 67a3924c4486991df5e4d38a995ff8054c145cf9
Summary: This is an sqlite equivalent of what exists in xdb now.
Reviewed By: krallin
Differential Revision: D21427621
fbshipit-source-id: 7024fbf7a8773c4465d2e6ee327aadeaf87cb213
Summary:
Implement autopull so non-automation `hg pull -r Dxxxx`, `hg up Dxxxx` will
pull `Dxxxx` automatically.
Since we now autopull the commits, error messages are removed. The old code
actually causes issues because it will raise at `"D1234" in repo`, which is
a surprise to many code paths, including the revset autopull logic, which
uses `x in repo` to decide whether `x` is unknown.
Note `hg pull -r Dxxxx` is not using the revset layer and needs to be handled
separately. It works for hg servers right now because the server can translate
`Dxxx` to a commit hash. It probably does not work for a Mononoke server.
Reviewed By: sfilipco
Differential Revision: D21320626
fbshipit-source-id: 939abe12e3a9a8ed5ca7ed29bb4f90fb39e7674a
Summary:
Change the interface to return infallible `Optional[node]` instead of fallible
`List[rev]`. In the next diff, we're going to use the 'node' information to
implement autopull.
Reviewed By: sfilipco
Differential Revision: D21320628
fbshipit-source-id: 6f7c070faba667cc85313cc78d6149c787ca8593
Summary:
Currently, phrevset picks the "successor" that has the maximum revision
number. That depends on the assumption that larger revision numbers
are modified last. Howevver, that's not always true with commit cloud sync. For example, in my repo I have D21179514 modified from 63768bf43
to 684612d5d. Phabricator has 63768bf43. Local successor is 684612d5d,
but the revision number of 684612d5d is smaller than 63768bf43:
quark@devvm1939 ~/hg/edenscm/hgext % hg log -r 'D21179514'
changeset: 63768bf436d01982a8d42ce97160ac6d9ae2cdad D21179514
user: Jun Wu <quark@fb.com>
date: Wed, 22 Apr 2020 09:45:50 -0700
summary: [hg] commitcloud: log metalog root during update references
quark@devvm1939 ~/hg/edenscm/hgext % lhg log -r 'D21179514'
changeset: 684612d5d606b01c224889f2b3f87aff7b93db49 D21179514 (@)
user: Jun Wu <quark@fb.com>
date: Wed, 22 Apr 2020 10:10:37 -0700
summary: [hg] commitcloud: log metalog root during update references
quark@devvm1939 ~/hg/edenscm/hgext % lhg log -r 'successors(63768bf436d0198
2a8d42ce97160ac6d9ae2cdad)' -T '{node} {rev}\n'
684612d5d606b01c224889f2b3f87aff7b93db49 76718
63768bf436d01982a8d42ce97160ac6d9ae2cdad 95363
Improve it by actually prefer selecting a non-obsoleted successor.
Reviewed By: sfilipco
Differential Revision: D21267552
fbshipit-source-id: d43d72a7c273c55af70bb41ad967fff0c78a452a
Summary: This is more efficient (if pullattempts have high chance to succeed).
Reviewed By: sfilipco
Differential Revision: D21320627
fbshipit-source-id: a2166f5a59f98c7d705c806b9d152ceb9981f3be
Summary:
Make it so that the autopull functions just describe what to do (using the
`pullattempt` struct) instead of having the side effect directly. This will be
useful in multiple cases:
- The actual autopull logic becomes easier to implement.
- The revset autopull layer can merge `pullattempt`s to fetch things in one go.
- The `pull -r` logic can reuse the `pullattempt`s to translate what to pull
(ex. from `D1234` to a commit hash).
This has subtle changes that multiple "remote"s are no longer properly
supported. That is probably fine in our production use-cases but we might
want to revisit if we want to support remotes "properly". Currently, this
greatly simplifies things due to the fact that "infinitepush" or
"infinitepushbookmark" have to be used in certain cases.
Reviewed By: sfilipco
Differential Revision: D21320633
fbshipit-source-id: e38b68abf69a34a97431685aa7ab0d2fe022fda8
Summary: Add a way for extensions to register how to auto pull unknown names.
Reviewed By: sfilipco
Differential Revision: D21320630
fbshipit-source-id: d20e11cff83b8a7e15a5085b0508a3e5bef305c3
Summary:
We're going to make autopull more complicated. Move it to a separate module as
it is not directly related to revset.
Reviewed By: sfilipco
Differential Revision: D21320631
fbshipit-source-id: fe60bc53ebf1c75f8bf66156805cbe2801fe6532
Summary:
The revset optimization makes unknown names harder to extract. For example,
`(or (list "a" "b" "c"))` will be optimized to `(_list "a\0b\0c")` and it
becomes harder to extract `"a"`, `"b"`, and `"c"`.
Move the unknown name extraction logic to before the optimization step to solve
it.
Reviewed By: sfilipco
Differential Revision: D21320632
fbshipit-source-id: 3a25f1cf4aab0449be6952113d622f29b1c0b631
Summary:
vs2017 is not able to compile the static assertion in KeySpace.cpp.
Previously we thought that this would be resolved in a later release of vs2017
but now that is here it is clear that it hasn't been fixed.
This commit pushes the version requirement to vs2019 (see
https://dev.to/yumetodo/list-of-mscver-and-mscfullver-8nd for a mapping between
product versions and compiler versions), but we cannot build with vs2019
because folly and rangev3 don't compile with vs2019, so this assertion (heh!)
has literally not been tested.
This commit also fixes up an oversight in the gating logic: the intent is that
we perform the assertion on all systems except known broken MSVC. We were
accidentally restricting it to later versions of MSVC.
Reviewed By: simpkins
Differential Revision: D21432890
fbshipit-source-id: e11ffccc53bf8dffdf2db45ad4f3cf199b1cc70d
Summary:
This allows us to understand what config is used during a transaction.
For example, is `selectivepull` enabled during a `pull`?
Reviewed By: DurhamG
Differential Revision: D21222146
fbshipit-source-id: a8c82f2b02e9657885947a706f728e28b1bfc1e2
Summary:
We're seeing deadlocks where if the process forks (like in a update
worker) while a background python thread is holding the progress lock, it can
cause a deadlock in the forked process if the thread ever tries to access the
progress bar.
To fix it, let's invalidate the engine if the process forks.
Reviewed By: xavierd
Differential Revision: D21415152
fbshipit-source-id: 75607dd2c1ed122b3a9df68d359ba9dcdde78a77
Summary:
D21364132 accidentally broke this; we can't run the fetcher
for projects for which we pulled the build out of cache, because there
is no source to update in that case.
This commit adjusts the logic so that we write out a marker file
to indicate that we installed a build from cache and to look for
that file being present to gate the new update logic.
Reviewed By: lnicco
Differential Revision: D21419122
fbshipit-source-id: 304670848add22531d88549d66f22c40ff255140
Summary:
There was a bug in scrub blobstore that caused failures while traversing the
fsnodes.
If all blobstores returned None, then we need to return None and not fail as we
do now. So the situation we ran into was:
1) fsnodes is not derived, all blobstore return None
2) Previously it returned the error, which later checked in
https://fburl.com/diffusion/mhhhnkxv - this check makes sure there's no entry
with the same key on the queue. However by that time fsnodes might already be
derived and someone else might insert a new entry in the blobstore and in the
queue. This would return an error to the client.
The fix here is to not fail if all blobstores returned None
Reviewed By: ahornby
Differential Revision: D21405418
fbshipit-source-id: 21fe130ce65a0087c408a5014e5b108c7ce8fe6c
Summary:
A number of repo names are used quite frequently. Let's use an enum to
prevent typos and make things cleaner.
Reviewed By: quark-zju
Differential Revision: D21365036
fbshipit-source-id: 1d3d681443df181e9076f5ee87029ae61124a486
Summary: Cover as much as remining code with `Cargo.toml`s, for the rest create an exlusion list in the autocargo config.
Reviewed By: krallin
Differential Revision: D21383620
fbshipit-source-id: 64cc78a38ce0ec482966f32a2963ab4939e20eba
