Summary:
This is a hook in mercurial, in Mononoke it will be part of the implementation. By default all non fastforward pushes are blocked, except when using the NON_FAST_FORWARD pushvar (--non-forward-move is also needed to circumvent client side restrictions). Additionally certain bookmarks (e.g. master) shouldn't be able to be moved in a non fastforward manner at all. This can be done by setting block_non_fast_forward field in config.
Pushrebase can only move the bookmark that is actually being pushrebased so we do not need to check whether it is a fastforward move (it always is)
Reviewed By: StanislavGlebik
Differential Revision: D14405696
fbshipit-source-id: 782b49c26a753918418e02c06dcfab76e3394dc1
Summary: There is no need to have an Option<Vec<XYZ>> as None can simply be represented by an empty vector. This makes these fields easier to use.
Reviewed By: StanislavGlebik
Differential Revision: D14405687
fbshipit-source-id: e4c5ba12a1e3c6a18130026af6814d54952da4d2
Summary: This is needed to run the scmadmin tool and lock the repo.
Reviewed By: StanislavGlebik
Differential Revision: D14497608
fbshipit-source-id: 5865b90375db29a17d462044ca4cdb87242a8209
Summary: To learn how far behind are we in the absolute bundle numbers.
Reviewed By: StanislavGlebik
Differential Revision: D14491672
fbshipit-source-id: 31d16f115b2b6fe4b88c25a847ce229e123b048b
Summary: We want to be able to manipulate hg-sync counters from the Mononoke admin.
Reviewed By: StanislavGlebik
Differential Revision: D14477676
fbshipit-source-id: 11218390bf469d4f297f7f13e9daee2d5f9bb35b
Summary:
Before adding hash validation to getpackv1 let's do this refactoring to make it
easier.
This diff also make hash validation more reliable. Previously we were
refetching the same file content again during validation instead of verifying the actual content
that was sent to the client. Since the content was in cache it was fine, but
it's better to check the same content that's sent to the client.
This diff also adds a integration test
Reviewed By: jsgf
Differential Revision: D14407292
fbshipit-source-id: b0667cb3dd6a7e0cee0b02cf87a61d43926d6058
Summary: Let's log it to scuba and to scribe just as we do with getfiles requests.
Reviewed By: jsgf
Differential Revision: D14404236
fbshipit-source-id: 079140372c128ee30e152c5626ef8f1127da36b1
Summary:
The diff adds support for getpackv1 request. Supporting this request is
necessary because hg client will stop using getfiles soon.
There are two main differences between getfiles and getpackv1. getfiles returns
loose files formats, while getpackv1 returns wirepack pack file (same format as
used by gettreepack). getpackv1 supports fetching more than one filenode per
file.
Differential Revision: D14404234
fbshipit-source-id: cfaef6a2ebb76da2df68c05e838d89690f56a9f0
Summary: The order was changed after ec00921aece0adc6aaca49e5580bff52784c4ca5
Reviewed By: quark-zju
Differential Revision: D14482440
fbshipit-source-id: d13327ed16387e597ca6fc1cfab0787a13f38d00
Summary:
We've see failures during the attempts to deploy this job to TW without such
waits. After adding the wait, things look better.
Reviewed By: StanislavGlebik
Differential Revision: D14456207
fbshipit-source-id: 8469932d7387060c026164ac97fa604453b0d296
Summary: Every packman-managed RPM in fbcode should now have a `packager: ONCALL` line in its packman.yml file. This diff adds one for fb-mononoke-admin. Assuming I found the right oncall, would you Accept+Ship this change?
Reviewed By: sunshowers
Differential Revision: D14399325
fbshipit-source-id: 5dd03b3f8722b4fb13b85acbe7702bc780b49f76
Summary: Increase timeout in line with observed operations.
Differential Revision: D14421000
fbshipit-source-id: 68941a5188e41c6dd7fbb3b59af0a912327f76a4
Summary: See D14279065, this diff is simply to clean up the deprecated code
Reviewed By: StanislavGlebik
Differential Revision: D14279210
fbshipit-source-id: 10801fb04ad533a80bb7a2f9dcdf3ee5906aa68d
Summary:
Currently we are logging new commits from BlobRepo. This will lead to issues once CommitCloud starts using Mononoke as we cannot differentiate between phases at that level. The solution is to log commits when they are pushrebased as this guarantees that they are public commits.
Note: This only introduces the new logic, cleaning up the existing implementation is part of D14279210
Reviewed By: StanislavGlebik
Differential Revision: D14279065
fbshipit-source-id: d714fae7164a8af815fc7716379ff0b7eb4826fb
Summary:
Fixes the below type error when pulling from a git repo using hg-git on my laptop:
File "edenscm/hgext/remotenames.py", line 225, in expull
pullremotenames(repo, remote, bookmarks)
File "edenscm/hgext/remotenames.py", line 314, in pullremotenames
path = activepath(repo.ui, remote)
File "edenscm/hgext/remotenames.py", line 1464, in activepath
rpath = _normalizeremote(remote.url)
File "edenscm/hgext/remotenames.py", line 1439, in _normalizeremote
u = util.url(remote)
File "edenscm/hgext/hggit/__init__.py", line 164, in _url
if not (path.startswith(pycompat.ossep) and ":" in path):
AttributeError: 'function' object has no attribute 'startswith'
Basically, `peer.url()` is the API, `peer._url` is a private field that does
not always exist somehow.
Besides, further remove named branches that can crash hg-git with
NotImplementedError:
File "edenscm/hgext/remotenames.py", line 225, in expull
pullremotenames(repo, remote, bookmarks)
File "edenscm/hgext/remotenames.py", line 322, in pullremotenames
for branch, nodes in remote.branchmap().iteritems():
File "edenscm/hgext/hggit/gitrepo.py", line 73, in branchmap
raise NotImplementedError
Reviewed By: DurhamG
Differential Revision: D14144462
fbshipit-source-id: 2e886c639cf6689480f84626eaf0d5ec25512ea0
Summary:
Apparently Gluster really can't copy with renames, so write to a
unique name, then symlink the canonical key to it. If the symlink already
exists, then we'll assume that the file already does, and remove the unique
name.
Reviewed By: StanislavGlebik
Differential Revision: D14014167
fbshipit-source-id: 1e5e2ce989652232d67d2aaac776e35127f58fb0
Summary: Keep track of error causes so we can print better errors.
Reviewed By: StanislavGlebik
Differential Revision: D14014165
fbshipit-source-id: e9a7846256bbfbfd689e0d78f01b1ac50cf64c1b
Summary:
Reduce directory fan out levels to 2 wide ones (xxx/xxx) rather than 4 narrow
(xx/xx/xx/xx). This saves write latency cost: with 4 levels, until we get
~billions of blobs, every write will also be a mkdir, and we'll have many
single-entry directories. With 2 wide levels this will be true up to ~millions of
blobs, but then start to fill out the leaf directories.
The downside is that the leaf directory loading will be higher once we do have
billions of objects, but I think it will be managable.
This is NOT INTEROPERABLE with 4-level fanout, so invalidates existing stores.
Reviewed By: aslpavel
Differential Revision: D13923991
fbshipit-source-id: b93ddc49305c921f8a1b34606796b6d75273fb75
Summary:
Adding `.data` to the end breaks the "rsync hack" to keep files colocated on
the same node.
Reviewed By: aslpavel
Differential Revision: D14014284
fbshipit-source-id: c65b237ed03c9c8ea3a6ac239135820379b7e38d
Summary: In this final diff for verify_integrity the data authorization data is being passed from hgcli (where the ssh connection is established) to Mononoke where it is further passed to verify_integrity script.
Reviewed By: StanislavGlebik
Differential Revision: D14387759
fbshipit-source-id: 2c0f9eef4128f5af0052276a1830ea2f449fb03a
Summary: The username and ssh vars are both properties of request, so they fit nicely inside per request CoreContext. They will be used in verify_integrity hook.
Reviewed By: StanislavGlebik
Differential Revision: D14385840
fbshipit-source-id: 9fe3cb96ffa89d8b017c730e37ca9ea4124ede0c
Summary:
This additional data that is send to Mononoke will be used e.g. in verify_integrity hook.
This diff also contains change to the user_unixname, instead of passing env variable USER there we pass value produced by users crate. Since the user_unixname is not used anywhere now it is safe to do so.
Reviewed By: StanislavGlebik
Differential Revision: D14385280
fbshipit-source-id: 1e48c232fafbba5d5188c7d162e9ef21efd738f7
Summary:
This stack is about adding getpackv1 wireproto request support (see task for
more details).
This diff adds a Decoder trait implementation. It parses all parameters for a
single file (i.e. filename + list of nodes). This Decoder is combined with
the previous diff to fully parse getpackv1 request
Reviewed By: lukaspiatkowski
Differential Revision: D14401516
fbshipit-source-id: 9b5fa1f48de338e58a288eb0653bd734cd8d1623
Summary:
The stack is about adding support for getpackv1 wireproto request.
This diff make decode_getfiles_arg_stream function generic over Decoder type.
At the moment decode_getfiles_arg_stream is used for unpacking `getfiles` wireproto
request. However getpackv1 packs request in a similar way to getfiles (which
is different from any other wireproto requests). Let's re-use the same
functionality for getpackv1
Reviewed By: lukaspiatkowski
Differential Revision: D14401517
fbshipit-source-id: ef0e9abeecec61c7c3d25b4a9e36da3f05c870cb
Summary:
Copy & rename sources must be included in the list of conflict files so that if
a copy was modified between root and `onto` bookmark then pushrebase should
fail with conflicts.
Note that some some merge cases are not handled yet - see TODO in the code
Reviewed By: lukaspiatkowski
Differential Revision: D14322036
fbshipit-source-id: d69bcceaa24987dd1e9d67e77f6a3205b580a7d8
Summary:
Mononoke does not support pushrebasing over a merge commit. Previously
`find_closest_ancestor_root` didn't detect merges.
In cases like
```
o <- onto
|
o o <- commit to pushrebase
|\ /
| o
o <- main branch
...
```
`find_closest_ancestor_root` would go to the main branch and finally fail with
`RootTooFarBehind` error. By detecting merge commit earlier we could print
better error message and avoid doing useless traversal
Reviewed By: lukaspiatkowski
Differential Revision: D14321616
fbshipit-source-id: 2aa53a2627f25897a241616a429864f1cfca3100
Summary:
The problem was in using `file_changes()` of a bonsai object. If a file
replaces a directory, then it just returns an added file, but not a removed
directory.
However `changed_entry_stream` didn't return an entry if just it's mode was changed (i.e. file became executable, or file became a symlink). This diff fixes it as well
Let's use the same computing changing files method instead of `file_changes()`.
Differential Revision: D14279470
fbshipit-source-id: 976b0abd93646f7d68137c83cb07a8564922ce17
Summary:
In integration tests, the sent replycaps part can be different lengths
depending on client capabilities. Usually this is globbed, but a couple
of places are missing.
Reviewed By: aslpavel
Differential Revision: D14363789
fbshipit-source-id: 5db161bd646c7a9fa8aeef2281ee7eb4d0c7771e
Summary: This endpoint is also used in Mercurial now.
Reviewed By: StanislavGlebik
Differential Revision: D14303557
fbshipit-source-id: fe38b62d010de2846dcf800f93ba050d9c396873
Summary: The phabricator message parsing capability will be also used from check_unittests Rust hook, so it had to be extracted and adjusted to Rust standards.
Reviewed By: StanislavGlebik
Differential Revision: D14301561
fbshipit-source-id: 47b59527dfadd7b761f750825da52ffce14fdf21
Summary:
There's a bug in pushrebase. When a directory is replaced with a file, then
changed file list is computed incorrectly. This diff adds a test that
highlights the issue, next diff fixes it.
Differential Revision: D14279472
fbshipit-source-id: dbb8738e4cd0b2c025060e91b5772662c296f342
Summary:
Make compute_changed_files accept manifest id, it will be used in the next
diffs
Differential Revision: D14279468
fbshipit-source-id: eca92900ed8862bb6db38ff6c5ed5372d8206aa9
Summary:
In the future we will be introducing mutation recording in commit extras. For
pushrebase, these fields need to be stripped when the commit is rebased. This
matches the Mercurial server behaviour.
At some point we will update Mononoke pushrebase to fill in these fields with
the pushrebase information, but that can wait until it is more widely deployed.
Reviewed By: quark-zju
Differential Revision: D14263856
fbshipit-source-id: 78b682fc580695137fd1fd4a4f65210a77f16b24
Summary: This diff adds a new `hg debuggethistory` command that takes filenode/path pairs from stdin, fetches the history of the files from the API server, and writes the results to a historypack in the hg cache.
Reviewed By: quark-zju
Differential Revision: D14248082
fbshipit-source-id: 8014a758abd3a578ea213d8d3177812629b2fd51
Summary: The Eden API client in Mercurial should be a singleton. This diff assigns the client to `repo.edenapi` so that it is accessible throughout the code.
Reviewed By: quark-zju
Differential Revision: D14233314
fbshipit-source-id: 8e0ed22c32611e8f6e7d4461c3e31870d47a0e95
Summary:
Client telemetry is used to send which commands are being run on the client,
which makes debugging slow sessions easier. For the client perspective, the
server hostname is send back so that the user knows to which physical host it's
connected, which is also useful debug info.
Reviewed By: StanislavGlebik
Differential Revision: D14261097
fbshipit-source-id: f4dc752671b76483f9dcb38aa4e3d16680087850
Summary:
We want to be able to reason about the progress of the sync job.
List of fields logged to [`mononoke_hg_sync`](https://our.intern.facebook.com/intern/scuba/query/?dataset=mononoke_hg_sync&pool=uber):
- reason
- entry
- repo_id
- succes
- err
- delay
- bookmark
Reviewed By: StanislavGlebik
Differential Revision: D14243842
fbshipit-source-id: 8ee3591be8f1dc9e1adca081b7b8eb51e7c6521a
Summary:
When converting from Mononoke `HgHistoryEntry` to Mercurial's `LooseHistoryEntry`, rather than directly copying the parents, we need to implement remotefilelog's logic for handling copied files.
Specifically, a copied file will usually have 0 parents (or 1 in the case of a merge). The filenode of the original (pre-copy) file is stored separately in the file's copy info. In the remtoefilelog loose file format, the filenode of the original file is stored as the p1 for the copy file, and the merge parent (if any) of the file is stored as the p2.
Reviewed By: StanislavGlebik
Differential Revision: D14256997
fbshipit-source-id: 09a070fb95d8b6c08c26e73156123d33cbfc685d
Summary: The code was accidentally setting the linknode for every entry to be the same as the filenode. This diff fixes the issue.
Differential Revision: D14251788
fbshipit-source-id: f0512c4c6ff59e2d22df0b42b4ee52a20f77a88f
Summary: Add an optional `depth` parameter to the `getfilehistory` endpoint, allowing the caller to specify the maximum number of history entries to fetch.
Differential Revision: D14222991
fbshipit-source-id: 8b28c9674e764f0098a9780f94ef2957ef117f09
