Summary:
Make `verify` check the lazy changelog properties:
- Universal id <-> name mappings are known locally.
- Segments are sane (ex. high-level segments are built from low-level and there
are no cycles)
With `--dag`, also check the graph with a newly cloned remote graph.
This just calls the verification logic added in Rust `dag` crate to do the
heavy lifting.
Differential Revision: D30820773
fbshipit-source-id: 8f62f41738c3c8e3fe88442860a83fdb4944f178
Summary: This proved useful couple of times when folks experienced problems with the agent.
Reviewed By: ahornby
Differential Revision: D30837676
fbshipit-source-id: aec769f60a09ecb83857e6e60d49a5662b4ce0b2
Summary: createremote only worked from root of the repo. This fixes it, and tests that in the integration test
Reviewed By: StanislavGlebik
Differential Revision: D30546582
fbshipit-source-id: 84aa304d346e448b44e5d7fb9e9607d84a67da25
Summary:
This adds basic logic for `snapshot restore` command.
- It updates to the parent of the snapshot
- It loads the snapshot changes
For now I did not do changes/tracked changes, as it will need to download the file contents, which will need a new edenapi endpoint, so I'll leave it for a future diff. It just restores your deleted files for now.
Reviewed By: StanislavGlebik
Differential Revision: D30543507
fbshipit-source-id: 080588ceff0ecd595ce739044f0d4118fb8e1a3f
Summary:
log sync reason for `hg cloud sync`.
This will help us to investigate issues better and measure impact for new Eden Api Uploads case by case (after amend, rebase, etc) on different platforms.
Reviewed By: yancouto
Differential Revision: D30775519
fbshipit-source-id: 696e954ec8db19226fb67ad0952e23e2b67e9931
Summary:
In the v1 sparse config arrangement, if all rules were excludes then we
would include a default "**" rule. This was always a little confusing and caused
some weird behavior. Let's remove it from the v2 world.
This actually bit us because the fbsource_exclude profile only has excludes,
which caused it to insert a ** include, which pulled in all of fbsource. We
could fix it to only check if a profile is excludes-only once all the transitive
profiles have been loaded, but I think the cleaner fix is to remove this logic
since it's confusing and never actually used in production.
Differential Revision: D30824082
fbshipit-source-id: adcf4c820cc9f7636f79759d03fc0b387b9f55fa
Summary:
Do not assume `changelog.parents` returns 2 items.
This changes the behavior for root commits. `parents()` used to return
`[repo[nullid]]`, now it returns `[]`.
Reviewed By: andll
Differential Revision: D30784684
fbshipit-source-id: 73f58c85457391fb74b96b88dc4dcb69a25e81ac
Summary:
In a future change, `ctx.parents()` returns `[]` instead of `[repo[nullid]]`
for root commits. Make the change to preserve absorb behavior.
Differential Revision: D30816385
fbshipit-source-id: afded91a6e72d4eb54faf87dcdfc52a81ea1d66f
Summary:
In a future change, `ctx.parents()` returns `[]` instead of `[repo[nullid]]`
for root commits. Make the change to preserve rebase behavior.
Differential Revision: D30816386
fbshipit-source-id: ca7c489991ae149c9640b7da0e6e54f76afbc250
Summary:
We're going to change parents() to return an empty list instead of `[nullctx]`
for roots. This change makes it more compatible with upcoming changes.
Reviewed By: andll
Differential Revision: D30787305
fbshipit-source-id: 1de523964faa64a6496a7bb0197af597e393d859
Summary: This will be used by the next change.
Reviewed By: andll
Differential Revision: D30784683
fbshipit-source-id: 59a37c5f428eaf5950584d8f17471d358bfefee7
Summary: Integrate http hash prefix lookup into the pull operation. One unfortunate change here is that if the prefix is ambiguous, we're only able to output possible full hashes as suggestions. Previously we'd also print commit log information. To retain that we'd need to add an error option to the response and have the server send back an error message with the log information or send another request to download the extra information.
Reviewed By: andll
Differential Revision: D30716050
fbshipit-source-id: 33f8bc38b0bfe7fce4ec11cd8def7feda3b3d3da
Summary:
It can sometimes be difficult to work out from the logging which commit cloud
requests came from which client repo. Previously you could often infer it from
the client identities, however if the request is proxied, the originating hostname can be
lost, and it still doesn't handle the case where the host contains multiple
repos.
This diff adds a new `ClientInfo` struct, which is included by the client
on every `get_references` and `update_references` request. This is logged
by the service, allowing us to correlate which client it came from, and what
workspace version the client had at that time.
Reviewed By: StanislavGlebik
Differential Revision: D30697839
fbshipit-source-id: 8fe2e03f0be2f443f8ae1814f083c04ba5d1805e
Summary:
This logic scans all the ancestors of the working copy that are not
ancestor of the graft source and checks their extras. With lazy changelog this
is extremely expensive. Let's just drop this logic.
Reviewed By: quark-zju
Differential Revision: D30734017
fbshipit-source-id: ca5606cea08efe10f29847970379d6bff4eb4aee
Summary:
When we remove a file from a sparse profile and commit the profile, it
should delete the file on disk. There's a bug where it doesn't actually delete
the file. This fixes it by passing the correct commit parents to the refresh
function.
Reviewed By: andll
Differential Revision: D30683677
fbshipit-source-id: 7f012faa99975d8270209f2962e7f9236890daed
Summary: This breaks all use of `hg sparse`, because `.hg*` cannot be matched.
Reviewed By: mitrandir77
Differential Revision: D30666349
fbshipit-source-id: c06d1b798a57490f2e5560f178a2839ae5425146
Summary:
We were copy/pasting metadata.get("version", "1") everywhere. Let's
make it a helper function.
Differential Revision: D30586162
fbshipit-source-id: ff6a9706f1970f84ffeb7de0e1362c3ba507fc00
Summary:
Sparse profiles should be roughly scoped around the files needed to
work on a certain product. If an engineer needs to work on multiple products
they should be able to enable multiple profiles.
Previously, multiple v2 profiles would be combined into an ordered list of
include/exclude rules, which meant that profiles enabled later could exclude
files included by the earlier profiles.
To fix this, let's treat each profile separately and create a matcher for each.
We then combine these into a union matcher, which means we're guaranteed to have
all the files that each profile specifies.
Differential Revision: D30586161
fbshipit-source-id: 2e04cfdba670ffce381a7c041706f315775ad7b0
Summary:
In a future diff we'll process sparse profiles differently at the
matcher creation level. To do so we need to expose the profile object to that
layer. Let's do it by storing the profile instead of just the profile name.
Differential Revision: D30586163
fbshipit-source-id: d90343b4101c43fbd838512289362aca7c3f816a
Summary:
The failpoint feature supports more complex injections, such as sleep, or fail
after a few times. There is no need to keep the adhoc faultinjection feature.
Differential Revision: D30495223
fbshipit-source-id: b5613811e489a5a52e9c0dd1ebf1096c848a402b
Summary:
This diff calls the `/:repo/snapshot` EdenApi endpoint added on D30514854 (ab17c4d181) from the `hg snapshot restore` command.
For now, it just prints the parent of the snapshot, but in next diffs it will update to it and restore the dirty changes.
Reviewed By: StanislavGlebik
Differential Revision: D30517984
fbshipit-source-id: e1381eaed561a7184ee02ab99d0282f11a1d944f
Summary:
I need to be able to run 'hg sparse files $PROFILE' in an Eden checkout
for a Sandcastle job that analyzes sparse profiles. We can't run 'hg sparse' at
all in Eden repos, so instead I run it in the backing repo. But that repo is
checked out to the null commit, so let's support --rev on 'hg sparse files' so I
can inspect an arbitrary rev.
Differential Revision: D30561529
fbshipit-source-id: 93b46caa9b63637bf4d63d5438bd23cbffb3983a
Summary:
Very basic shell for the `hg snapshot restore` command.
For now just reads the id and prints it to stdout. Next diffs will add functionality to it.
Reviewed By: StanislavGlebik
Differential Revision: D30449567
fbshipit-source-id: 9f4e0bb284dff7e6ffb4398942f3247a09511933
Summary:
fburl.com/botsdiffs is a random diff
fburl.com/botdiffs is the wiki
Differential Revision: D30547647
fbshipit-source-id: 337d6457cb6403f11fbbc9654f3d34f50d69b0e5
Summary: Generalise the hook slightly; instead of relying on a specified file, use a revset, which opens this out to more use cases (e.g. blocking revisions in fbsource that aren't on our `master` branch)
Reviewed By: StanislavGlebik
Differential Revision: D30511155
fbshipit-source-id: 6b73c7c3e6caf2d670632110619eacb7b6216355
Summary:
Adds a helper "hg debugsparseprofilev2 $PROFILE" command that prints
out the difference in files matched by the profile using v1 and v2. This is
useful when migrating a profile to v2 and checking you haven't broken anything.
Reviewed By: kulshrax
Differential Revision: D30303868
fbshipit-source-id: 1b1d8b197470692fa13457ecb4c9344104b6d9c0
Summary:
v1 sparse profiles had excludes always take precedence over includes.
This made it hard to use and resulted in sparse profiles that were too wide.
Let's add a v2 sparse profile that respects the order of include/exclude rules.
Reviewed By: andll
Differential Revision: D30284698
fbshipit-source-id: 6765b8487695ec6cc7f8027e4da98ca65957b5d0
Summary:
In a future diff we'll want to handle ordered lists of include/exclude
rules. To do so we need to use the treematcher. Note, this means we can't use
regular expressions for sparse matchers now, but there's only one occurrence of
that in production.
Reviewed By: andll
Differential Revision: D30284700
fbshipit-source-id: c379e1feb233baee8c5ba2e1f57017b72ecaa835
Summary:
Currently excludes always take precedence over includes in sparse
configs. In a later diff we want to make this more flexible. To start with
though, let's refactor config loading to use a more "rules" oriented structure.
Excludes will still take precedence over includes, but will do so by being later
in the rule list. This will allow us to start changing rule ordering in a later
diff.
Reviewed By: andll
Differential Revision: D30284699
fbshipit-source-id: 955c016d67e22fce6b26ba7b8e1ffacc931989c8
Summary:
Previously the SparseConfig type was abused to mean three things:
1. The non-recursive contents of .hg/sparse
2. The non-recursive contents of a sparse profile.
3. The recursively expanded contents of enabled sparse rules and profiles.
In a later diff we'll be changing how includes/excludes work to make it more
flexible, and this reuse of the same type makes it complicated.
So let's split it into three distinct types. Later diffs will change them so
they aren't so similar. In doing so, we also slightly change the way sparse
profiles are loaded, to load them recursively.
Reviewed By: andll
Differential Revision: D30284701
fbshipit-source-id: f908ba167a04f9863ab75e1cd6a94027fa64607e
Summary:
This fixes a small bug in snapshots, as noted in the `TODO` comment I deleted.
If you `hg rm` a file and then replace it with something else, we want that to be uploaded to the snapshot as well.
These types of files are uploaded as "UntrackedChange", which is a bit confusing but correct, it means the file was deleted and now there are untracked changes in it (similar to how if a file is marked as "untracked deletion" but it didn't exist, it means it was `hg add`ed and then locally deleted)
Differential Revision: D30398143
fbshipit-source-id: dbd316fbcdc0e9b1781c5c967eb12c02805f4361
Summary:
Before this diff, only file changes (tracked or untracked) were stored in the changeset.
This diff makes it so deletions and untracked deletions are also stored in the changeset.
Differential Revision: D30373960
fbshipit-source-id: 4c1c40f4028e03a7858ce7ce55d385e02b672fc4
Summary:
This allows us to get more insights on race condition (ex. pull reverts part of
what cloud sync did) issues.
Reviewed By: andll
Differential Revision: D30415135
fbshipit-source-id: c99ce77d2748e503aea523e485be5b7a57ee8b98
Summary:
Many diffs include multiple tags as prefixes, however the current implementation only greedily takes prefix as the substring up to ']'.
## Issues
Current behaviour for
- **non-tag right bracket:** `My fake diff title with [link]()` -> `My fake diff title with [link]`
- **multiple tags:** `[hg][extensions] fake diff title` -> `[hg]`
## Solution
Use regex to capture all prefix tags:
```
(?:\[.*?\])+
```
## Explanation
- Non capturing group ( `(?: ... )` )
- matching the open bracket ( `\[` )
- and any character as few times as necessary (lazy) ( `.*?` )
- and the closing bracket ( `\]` )
- matching the group as many times as needed (greedy) ( `+` )
Also note `re.match()` matches from the beginning of the string (unlike `re.search()`)
Reviewed By: ronmrdechai
Differential Revision: D30415867
fbshipit-source-id: e09d4e6d2759d0106d41d1a5d4e607ec34eef3fa
Summary:
It's `O(tr.get('nodes'))`, which does not scale. With BFS prefetch, it's not
that slow if the tree isn't prefetched. It has already been disabled for major
repos D23912965, and causes slowness with lazy changelog (see below).
Reviewed By: StanislavGlebik
Differential Revision: D30454407
fbshipit-source-id: 8027b5e5f1ee09a5f1ffe98a638585345464dd3d
Summary: The nth ancestor revset was crashing with multiple base revisions, e.g. "(. + .^)~". Fix variable shadowing issue in revset.ancestorspec.
Differential Revision: D30375233
fbshipit-source-id: 37a78bf1000a40872600e587733a84029f68343b
Summary:
For large megarepos that are result of merging multiple smaller repos
navigating to pre-merge history is usually not what user wants. The checkout
will be slow (in non-edenfs repos), the tooling that expects some repo
structure won't work etc.
Reviewed By: StanislavGlebik
Differential Revision: D30394205
fbshipit-source-id: 23fc4fc31bf01d4cc14f6e3baa1e1165a26a1896
Summary: Will be used in the next change.
Reviewed By: andll
Differential Revision: D30314053
fbshipit-source-id: 6491bcfe8b5667075521d021b995e71aab08d7e7
Summary:
This diff makes the snapshot command upload all types of files that were missing (added/untracked/missing/deleted), using the new types of file changes added on the previous diff.
Next steps:
- Add some indicator to Bonsai Changeset saying it is a snapshot. Verify only snapshots can have certain file changes (untracked/missing).
- Upload the files and the changeset inside an ephemeral bubble instead of in the main blobstore
- Start writing the `snapshot restore` command
Differential Revision: D30137673
fbshipit-source-id: 555238f1d64a5438cde35a843043884a939de4fe
Summary:
On Windows, it's hard to rename `.hg/store/segments/v1` away because the files
are (too easily) being currently used. Instead, let's write the new lazy
segments to `segments/v1next`, and try to rename it back later on follow-up
commands.
Reviewed By: DurhamG
Differential Revision: D30118952
fbshipit-source-id: e3edb588dccf1acb5f4ed106bbb979bcc8c6e67e
Summary:
OD was running `hg rage --preview` in the background periodically and that has caused edenfsctl creating paste on the user's behalf without them actually knowing.
This will make Mercurial to collect EdenFS rage in dry-run mode (i.e. do not create paste).
Reviewed By: quark-zju
Differential Revision: D30288390
fbshipit-source-id: 1e7f25648ca1b76f24264ee13ed98cf974148b0f
Summary: Previously, contentstore metrics were tracked as a special case. Now that I'm tracking metrics on each backend, I've standardized the contentstore metrics to match.
Reviewed By: DurhamG
Differential Revision: D29995500
fbshipit-source-id: 3133dca5912d9cd6c26bd88d9aeb6a572f6eca40
Summary: Clean Up: Once we migrated to Mononoke we can remove the infinitepush path because it creates extra complexity in remotenames, infinitepush and commitcloud extensions
Reviewed By: yancouto
Differential Revision: D30277402
fbshipit-source-id: 1708a71eb33dc9a9b509bdc1b03bed08073b6152
Summary:
fix remote lookup for every bookmark
Some users have lots of local bookmarks and remote bookmarks, so the current logic would perform tons of sequential remote lookups via edenapi (due to segmented changelog) just to maintain correct omission state for local and remote bookmarks even if nothing has been changed.
Reviewed By: markbt
Differential Revision: D30247185
fbshipit-source-id: 7c7fd8491ff0bd67827c2a01eb852322ba15f994
Summary: This allows us to remove older files to free disk space.
Reviewed By: DurhamG
Differential Revision: D30092445
fbshipit-source-id: 20770968cfaeba4fb6f6c45b247cc1cdafb79a08
Summary: This makes upcoming changes a bit easier.
Reviewed By: yancouto
Differential Revision: D30092446
fbshipit-source-id: f62f1ced45c46dc48c3b1112af661d8fbcd80706
