Summary: Make `parents`, `data`, and `metadata` optional, and introduce `WireTreeAttributesRequest` for selecting which attributes to request on the wire.
Reviewed By: kulshrax
Differential Revision: D23406763
fbshipit-source-id: 5edd674d9ba5d37c23b12ab4d7b54bbf6c9ff990
Summary:
Adds a `WireTreeQuery` enum for query method, with a single `ByKeys(WireTreeKeyQuery)` available currently, to request a specific set of keys.
Leave the API struct alone for now.
Reviewed By: kulshrax
Differential Revision: D23402366
fbshipit-source-id: 19cd8066afd9f14c7e5f718f7583d1e2b9ffac02
Summary: The size can change with zstd upgrades. Do not test them.
Reviewed By: sfilipco
Differential Revision: D23976933
fbshipit-source-id: d560061b6e4fefc3bb89513bdb12c770ea0bd881
Summary:
This diff introduces Mysql client for Rust to Mononoke as a one more backend in the same row with raw xdb connections and myrouter. So now Mononoke can use new Mysql client connections instead of Myrouter.
To run Mononoke with the new backend, pass `--use-mysql-client` options (conflicts with `--myrouter-port`).
I also added a new target for integration tests, which runs mysql tests using mysql client.
Now to run mysql tests using raw xdb connections, you can use `mononoke/tests/integration:integration-mysql-raw-xdb` and using mysql client `mononoke/tests/integration:integration-mysql`
Reviewed By: ahornby
Differential Revision: D23213228
fbshipit-source-id: c124ccb15747edb17ed94cdad2c6f7703d3bf1a2
Summary:
This makes it so commit hashes are serialized to bytes instead of tuples in Python:
In [1]: s,f=api.commitdata(repo.name, list(repo.nodes('master')))
In [2]: list(s)
Out[3]: [{'hgid': '...', ...}]
Some `Vec<HgId>`s cannot be changed using this way. It'd be nice if we can change
the default `HgId` serialization to bytes.
Reviewed By: kulshrax
Differential Revision: D23966989
fbshipit-source-id: 4d013525419741d3c5c23621be16e70441bab3c4
Summary:
`HgId` currently serializes into a tuple of 20 items. This is suboptimal in
CBOR, because the items are untyped. A byte might be serialized into one or two
bytes:
In [2]: cbor.dumps([1,1,1,1])
Out[2]: b'\x84\x01\x01\x01\x01'
In [3]: cbor.dumps([255,255,255,255])
Out[3]: b'\x84\x18\xff\x18\xff\x18\xff\x18\xff'
CBOR supports "bytes" type to efficiently encode a `[u8]`:
In [5]: cbor.dumps(b"\x01\x01\x01\x01")
Out[5]: b'D\x01\x01\x01\x01'
In [6]: cbor.dumps(b"\xff\xff\xff\xff")
Out[6]: b'D\xff\xff\xff\xff'
Add `serde_with` with 3 flavors: `bytes`, `tuple`, `hex` to satisfy different
needs. Check the added docstring for details.
Reviewed By: kulshrax
Differential Revision: D23966992
fbshipit-source-id: 704132648f9e50b952ffde0e96ee2106f2f2fbcf
Summary: The logs would never show what directory was changed to be a placeholder.
Reviewed By: fanzeyi
Differential Revision: D23977719
fbshipit-source-id: b1f7f539457956510638a31ca3e49c4cee641fd8
Summary: This is not needed, we can use the PrjfsChannel class directly where needed.
Reviewed By: chadaustin
Differential Revision: D23946259
fbshipit-source-id: eafcd38c0927fa282d62ada0986a7ef8b612174b
Summary: It seems after updating zstd to 1.4.5. `Dllexport` for zstd.dll was not being picked up correctly. Instead of having zstd being a runtime dependency let's try statically link it to avoid the DLL issue.
Reviewed By: vitaut
Differential Revision: D23970349
fbshipit-source-id: 3b14dddb64d410cb9546c416f27d73b7604b21ba
Summary: We use hyphens in other bypass strings. Make this consistent in `test-hooks.t` to avoid confusion.
Reviewed By: mitrandir77
Differential Revision: D23964799
fbshipit-source-id: e300bad091aa6c50f5921507117c1019b9863bd5
Summary:
Let's start actually fixing what commit sync config version is used to remap a commit i.e. we
should use a commit sync config version that was used to remap a parent instead
of using a current version. See more details in
https://fb.quip.com/VYqAArwP0nr1
This diff fixes one particular case and also leaves a few TODOs that we need to
do later
Reviewed By: krallin
Differential Revision: D23953213
fbshipit-source-id: 021da04b0f431767fec5d1c4408287870cb83de1
Summary:
TestLiveCommitSyncConfig is supposed to be a test replacement of
CfgrLiveCommitSyncConfig, however it was quite a different semantic. In
particular, it wasn't even possible to have two versions of the mapping for the
single repo.
This diff changes that. Now we'll have a method to add commit sync config
version, and mark/remove a version as current
Reviewed By: krallin
Differential Revision: D23951202
fbshipit-source-id: 242b4f088f67dac504544987e484cc290ee4e400
Summary: Instead of always fetching the current version name to verify working copy let's instead fetch whatever the version was actually used to create this commit.
Reviewed By: krallin
Differential Revision: D23936503
fbshipit-source-id: 811e427eb62741401b866970b4a0de0c1753edb3
Summary:
Turned out validation didn't report an error if source repo contained an entry
that was supposed to be present in target repo but was actually missing.
This diff fixes it.
Reviewed By: krallin
Differential Revision: D23949909
fbshipit-source-id: 17813b4ad924470c2e8dcd9d3dc0852c79473c61
Summary: Since now we store it in the db, let's also expose it in CommitSyncOutcome enum
Reviewed By: krallin
Differential Revision: D23936502
fbshipit-source-id: a0758143ceaa8f5706f1d9cfe3040ac91c7bac49
Summary: This just adds some explanation of `-r` to the `hg sparse list` docstring.
Reviewed By: markbt
Differential Revision: D23961027
fbshipit-source-id: 64ab406b07fe5d66fd53d4e520935aad3b0b351b
Summary:
Dynamicconfig can generate configs two ways, 1) via `hg
debugdynamicconfig` and 2) synchronously in-process in an hg command when it
detects that the dynamicconfig is completely missing or has the wrong version
number.
In the first case, dynamicconfig gets the repo name from the standard config
object loaded by the hg dispatch. In the second case, the standard config
object isn't even loaded yet, so dynamicconfig does a mini-load of the user and
repo hgrcs so it can get the repo name and user name (needed for dynamic
conditions).
Unfortunately the second code path computed the wrong path (it had two .hg/'s)
which meant the reponame and user name were always none. This meant that the
dynamicconfig on disk could randomly be either computed with or without a
reponame.
Let's fix the path computation, and add a test. We may want to make
dynamicconfig fail if no repo name is passed, but I'm not sure if we'll want to
support no-repo configuration at some point.
This didn't cause a problem for most people, since it would only happen during a
hg version number change, and 15 minutes later the background 'hg
debugdynamiconfig' process would fix it up. It did affect sandcastle though,
since it often creates new repositories and acts on them immediately.
Reviewed By: quark-zju
Differential Revision: D23955628
fbshipit-source-id: c922f4b523d19df9223aa28c97700b7011fc03eb
Summary:
The old code tried to express 4GB by using ^ to do an exponent. That
operator is actually the bitwise xor, so this was producing a limit closer to 4
bytes. It doesn't seem to have mattered much since a later diff overrode the
default via dynamicconfig. But let's fix this anyway.
Reviewed By: krallin
Differential Revision: D23955629
fbshipit-source-id: 6abebcb7e84b7a47f70ac501fa11b0dc60dfda7b
Summary: The diff adds API to create a set of connections: read, read master and write.
Reviewed By: ahornby
Differential Revision: D23568561
fbshipit-source-id: b3ee954604557497ed56c6b369256b6f76a1e042
Summary:
This diff makes blobstore healer to use MyAdmin to get replication lag for a DB shard and removes "laggable" interface for connections.
The old "laggable" API worked this way: we maintained potential connections to each possible region, then tried to query replica status on all of them. If there was no replica hosts in some of the regions, we just wanted to ignore it by handling a specific error type.
This is legacy and makes the logic more complicated. We want for the new code to use Myadmin instead.
Reviewed By: krallin
Differential Revision: D23767442
fbshipit-source-id: 9f85f07bd318ad020d203d2bcd1c8898061f7572
Summary:
I spotted D23955629 recently, and it turns out I made the same mistake in LFS.
Let's fix it. In order to keep the same semantics (yield on every chunk from
the filestore), I lowered the threshold a bit from what it was intended to be.
Reviewed By: johansglock
Differential Revision: D23960152
fbshipit-source-id: 4d384752228fd125ade7e032a477648798e7fa44
Summary: Now that the `async_runtime` crate exists, use Mercurial's global `tokio::Runtime` instead of creating one for each EdenAPI store.
Reviewed By: quark-zju
Differential Revision: D23945569
fbshipit-source-id: 7d7ef6efbb554ca80131daeeb2467e57bbda6e72
Summary:
Pull Request resolved: https://github.com/facebookexperimental/rust-shed/pull/12
The OpenSSL version on Mac doesn't work well with EdenSCM and Mononoke integration, just use the one from getdeps/brew.
Also remove the now redundant "DEVELOPER_DIR" since the modern XCode version works.
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/63
Reviewed By: StanislavGlebik
Differential Revision: D23927022
Pulled By: lukaspiatkowski
fbshipit-source-id: 6b6b3baa33d49b567b9aa6178cbd20b7ae9edc89
Summary:
This is (I think) the last step required to make rust-partial-io be github-first.
The diff was created using:
* `zbgs partial-io` and remove all instances of it in fbsource
* `hg rm common/rust/partial-io`
* add `partial-io` to `third-party/rust/Cargo.toml`
* `common/rust/tools/reindeer/vendor` to vendor
* `buck build eden/mononoke/mercurial/...` to check that it builds correctly
* `buck run //common/rust/cargo_from_buck:cargo_from_buck` to run autocargo
Reviewed By: aslpavel
Differential Revision: D23849634
fbshipit-source-id: 339fc3976cc9a0b6f10a0538d643b87797e2bc3c
Summary:
`arc install-certificate` is going away, so let's reference the replacement
Created from Diffusion's 'Open in Editor' feature.
Differential Revision: D23948479
fbshipit-source-id: e629458ed2be1e1c09d9674935e3bcc890b05ad9
Summary:
See motivation for the change in D23845720 (5de500bb99).
We'll need to store version name even for commits that weren't rewritten, but that have an equivalent working copy in another repo.
Reviewed By: ikostia
Differential Revision: D23864571
fbshipit-source-id: 408b68c3b0aa9885a9cd248b0b4abc2b87cd4cca
Summary:
get_source_target_mover likely awaits the same fate as
get_current_mover_DEPRECATED functions i.e. get_source_target_mover will likely
be removed.
This diff just removes a few intances of this function.
Reviewed By: ikostia
Differential Revision: D23929748
fbshipit-source-id: 2ac09da164de3916a552757acf0c39387f6126e4
Summary:
get_mover() and get_reverse_mover() functions return the mover for the
"current" version of the commit sync config, which means these are movers for the version
of the config that's used to create the latest commits on master branch.
So this function returns correct mover only for the latest master commit, but
for all other commits it returns an incorrect mover! This is wrong and it
happened to work just by change, and that's why these functions are marked as deprecated
now, and later we'll add functions 'get_mover_by_version()' which could be used to
replace deprecated functions.
Note that the story for get_bookmark_renamer()/get_reverse_bookmark_renamer()
functions seems to be different. If we can always figure out what's the correct
mover for a commit by e.g. look at its parent we can't really do the same for
bookmarks. Because of that I suggest to keep using the current version for
get_bookmark_renamer() function.
Reviewed By: ikostia
Differential Revision: D23929582
fbshipit-source-id: 3e5e9b46224aca0b75cf2d981ea21c4f9a378ba9
Summary: Finally remove version_name from CommitSyncRepos. Note that this diff adds a few TODOs that we'd need to fix later.
Reviewed By: ikostia
Differential Revision: D23929010
fbshipit-source-id: c72130af548ac7b26bc20ddaac9a59562cc75e0b
Summary: Just as in the previous diff, but this time remove bookmark_renamers from CommitSyncRepos
Reviewed By: ikostia
Differential Revision: D23910295
fbshipit-source-id: 0c2d147057c8d3e0749d5b31ef98ab5022255d95
Summary: Just as the previous diff, but this time it removes reverse_mover
Reviewed By: ikostia
Differential Revision: D23879509
fbshipit-source-id: ed111ca2d106120229c4facc0bb2435913c27966
Summary:
This diff starts to use CommitSyncDataProvider introduced in the previous diff
and removes Mover from CommitSyncRepos struct.
Reviewed By: ikostia
Differential Revision: D23878683
fbshipit-source-id: 0d54f889781aebe4726b3388343a87df783c17d4
Summary:
As described in D23845720 (5de500bb99) we are doing a pretty significant change in the
CommitSyncer. Previously it stored static Movers and BookmarkRenamers, but it
needs to change and they would need to fetch config from LiveCommitSyncConfig.
Unfortunately we already have a bunch of tests that create weird movers that
would be very hard to model via LiveCommitSyncConfig. So we have 2 options:
1) delete or rewrite all these tests
2) Create a wrapper that would return a mover/bookmark renamers.
Option #2 is preferable, and that's what this diff does. In production it would
use LiveCommitSyncConfig, but it also let's tests specify whatever weird movers
they need.
Reviewed By: ikostia
Differential Revision: D23909432
fbshipit-source-id: 83fb627812f625e07f7e40044e2f69274cd2d768
Summary:
Now that public phases are separated from draft, hidden behind the
ispublic() api, and no consumers require the entire set anymore, let's make them
lazy by testing commits against a lazy ancestor collection.
Reviewed By: quark-zju
Differential Revision: D23921326
fbshipit-source-id: 7c53345f41ff89d2d0ec11b1c09bbb2f89529c21
Summary:
Previously, _genrevancestor applied it's cutfunc function (which is
usually used to bound an ancestor traversal from processing public nodes) when
producing the parents of the current rev. In the walkrevtree function this means
we have to invoke cutfunc before we enqueue new parents onto the heap. This can
mean we have to process very old commits, even though walkrevtree might never
get to them.
Let's move the cutfunc to be at the deque stage instead of enque. When we make
public phases lazy, this will let us avoid looking up really old commits and
speed things up a lot.
Reviewed By: quark-zju
Differential Revision: D23921324
fbshipit-source-id: 8b520e9184441b9bd7581cdb71414f406cf9b544
Summary:
Unfortunately the current add operator implementation for nameset sorts
the result, which causes lazy namesets to be evaluated. Let's change the phase
revset to avoid that operator for now.
In a future diff we'll make publicrevs lazy, which will require this change to
be performant.
Reviewed By: quark-zju
Differential Revision: D23921325
fbshipit-source-id: c3477c30c1c5061b641c14b8aa73d07da6f7f398
Summary:
In a future diff we'll be changing the way public phase computation
works. Let's start by moving the draft phase computation out to belong to
draftrevs(), so then we can change publicrevs() without worrying about draft. In
the process, let's simplify draftrevs to just be an only() query.
Reviewed By: quark-zju
Differential Revision: D23921327
fbshipit-source-id: 255eab4b796bf7bc4467544cf8231b3ed98c08c8
Summary:
In an upcoming diff we'll be splitting draft and public calculations
apart when in headbased repositories. To make this cleaner, let's stop using
loadphaserevs and start using draftrevs and publicrevs accessors.
Reviewed By: quark-zju
Differential Revision: D23921328
fbshipit-source-id: 26a2a6fdc1201416bf61c7d32e3ede62d9579546
Summary:
Our configerator configs store both "current" mapping version and also they
store all versions that were used before.
These integration tests were updating just current version, but weren't
updating "all" versions. This is incorrect but it worked by accident.
But it will stop working in the next diffs, so this diff fixes it
Reviewed By: ikostia
Differential Revision: D23908970
fbshipit-source-id: 10f96bd02987d9195aff4855241efbd9a065a761
Summary:
A higher-level goal is to provide an interface for the manual remediation of
various xrepo-sync blockages. The nature of a candidate selection is such that
it may fail if the hint is not sufficient to decide which remapping changeset
is the best candidate. This is especially true about the `Only` hint variant:
it is designed to fail when there's more than one candidate. But even with
bookmark ancestorship hints, there are corner cases when the algorithm cannot
make a decision (as well as there may just be bugs in the algorithm). These
cases should be **extremely** rare. Nevertheless, we want to be able to unblock
ourselves. To do so, it is proposed to acccept parent selection hints in the `xrepo-lookup` scs
method. By default, it will use `Only` as a hint and be semantically equivalent
to the current behavior. But we'll provide CLI options to select other hints.
In order to make this work, we need the `sync_commit` method of the
`CommitSyncer` to accept hints too.
Reviewed By: StanislavGlebik
Differential Revision: D23913216
fbshipit-source-id: 05e1ff99cd2c6522829a6e8569040b226600af60
