Summary:
This allows the streampager to be configured via hgrc files.
Default are picked so the behavior is closer to the current default pager
(`less -FRX`).
Reviewed By: DurhamG
Differential Revision: D20902034
fbshipit-source-id: 994ab963ceace02eeb1d18cfa5768e411ca3610b
Summary:
The Rust worker code will only work with remotefilelog, and when the Rust
ContentStore is enabled, make sure to not enable it if these 2 conditions
aren't met.
Reviewed By: quark-zju
Differential Revision: D21033428
fbshipit-source-id: c34c1b39ddb81be399463712216fa2cd68771f41
Summary:
Computing delta force the client to have the previous version locally, which it
may not have, forcing a full fetch of the blob, to then compute a delta. Since
delta are a way to save on bandwidth usage, fetching a blob to compute it
negate its benefits.
Reviewed By: DurhamG
Differential Revision: D20999424
fbshipit-source-id: ae958bb71e6a16cfc77f9ccebd82eec00ffda0db
Summary: D21017515 removed the `since-last-arc-diff` flag; this was the last reference to it in the hg codebase.
Reviewed By: singhsrb
Differential Revision: D21025752
fbshipit-source-id: 083cb821263fa844ec74773d8636e19a6e82c550
Summary:
I noticed we removed the call to getManifestEntry, but didn't remove
the method from the EdenThriftClient.
Reviewed By: singhsrb
Differential Revision: D21025523
fbshipit-source-id: 409eb52bad08b305ecfd0f8918cb71fec6a71aab
Summary:
This flag is an obsolete duplicate of since-last-submit.
Eliminate it to facilitate convenient tab autocompletion and make
the world less complicated.
Reviewed By: xavierd
Differential Revision: D21017515
fbshipit-source-id: f3e4db602a69f1c7da886d455ac36c4a2ddb6b01
Summary:
Having some colors, even dull is better than no colors. For every "bright"
color found in the repo, also specify their non-bright variant.
Reviewed By: DurhamG
Differential Revision: D20972198
fbshipit-source-id: ca10c0eac13129f120fc45623d134f279a75b3c2
Summary: We're writing utf-8 data to it, decode it before returning it.
Reviewed By: DurhamG
Differential Revision: D20972741
fbshipit-source-id: fb23a6d1c3188e10598b839639cdd5e74322089a
Summary:
While keys are strings, values are bytes buffer and thus needs to be converted
sometimes.
Reviewed By: DurhamG
Differential Revision: D20974484
fbshipit-source-id: 13394f5dc43191e85e4b1d350cc4fbbd8489572a
Summary:
Symlinks are pointers to path, that are required to be utf-8, thus we can
decode the data.
Reviewed By: DurhamG
Differential Revision: D20973701
fbshipit-source-id: 8399dfb3462fea907758a01ed2fcea508833fbaa
Summary: `hyperlink(url, title)` returns a terminal escape sequence for the url even if the title provided is empty. While the terminal displays nothing, the actual string in hg-template-land is non-empty (it contains a url), and all the template functions that check for empty strings don't see it as empty.
Reviewed By: fanzeyi, xavierd
Differential Revision: D20970733
fbshipit-source-id: 04436714f3827b98b8fb249b3eb1da82f366807c
Summary: In Python3, str is not a byte string, but a unicode one. Use bytes instead.
Reviewed By: sfilipco
Differential Revision: D20965794
fbshipit-source-id: b04ff004aac87fd3afc8070fc8d1d849ad48d0d3
Summary:
In order to grep, the body of the file is decoded to utf8 with the error
handling "replace". This is probably OK as I would expect histgrep to be used
on text files, not random binary ones where grepping would probably not make a
lot of sense.
Reviewed By: sfilipco
Differential Revision: D20965178
fbshipit-source-id: 9ba234f33e801a78537c22b61dee434625449c01
Summary: The file is already written with writeutf8, read should use readutf8 too.
Reviewed By: sfilipco
Differential Revision: D20949694
fbshipit-source-id: cb00f45313ba8f78197d2344db7adc787b13ee4d
Summary:
Dulwich expects pretty much everything to be bytes, thus we have to convert a
lot of stuff to bytes from unicode. This only fixes one test, but makes
progress on all of them.
Reviewed By: sfilipco
Differential Revision: D20949172
fbshipit-source-id: 44cd0c5136b3d224f7eb81e48ec98fed60c8b390
Summary:
These classes were used for upstream's treemanifest implementation. At
one point we partially used it for our server side tree logic, but these days we
don't use any of it except for the revlog layer. Let's delete the context and
treemanifest layers and tests.
Reviewed By: sfilipco
Differential Revision: D20953163
fbshipit-source-id: 952063e4e0cf74e2da740bf91a70e1714a9ad9b5
Summary:
Disables 'hg serve' being able to start a web server. This can be
worked around temporarily by setting web.allowhgweb=True, but we should remove
that workaround soon so we can deleted all the code.
Reviewed By: xavierd
Differential Revision: D20951800
fbshipit-source-id: e0d5017647a073b5ede3b0bbc4dd745fabeac816
Summary: The .keys() in Python3 returns an iterator, hence '+' doesn't work on these.
Reviewed By: quark-zju
Differential Revision: D20956491
fbshipit-source-id: 15c74bfcfe6d27dff5018fdf71e61bb747daf2d8
Summary:
The iteritems method doesn't exist in Python3, add the items one and remove the
pycompat.itermitems wherever we iterate over the lazyremotenamedict.
Reviewed By: quark-zju
Differential Revision: D20926027
fbshipit-source-id: de9d297126e8190f008cb5d09cc5a128d68c2213
Summary:
Drop other diffhelpers implementations.
The Rust implementation does a better job on type checks and won't segfault.
Reviewed By: xavierd
Differential Revision: D20935367
fbshipit-source-id: f59240f3a661efe1ca6e451922bcb3ca6ee6ae22
Summary:
This largely reverts the changes in D9443604, to make `hg rage` report the
full configs again.
Our configs vary from tier to tier and repository to repository. The override
settings alone don't seem like enough to try and figure out what the actual
settings are that the user is using.
I retained the behavior of reporting the source file and line number for each
config item, which wasn't present before D9443604. I also still excluded
config values from the "builtin" config source. Currently this only excludes
the handful of merge-tools defined in
`eden/scm/edenscmnative/bindings/modules/pyconfigparser/src/lib.rs`
Reviewed By: quark-zju
Differential Revision: D20927599
fbshipit-source-id: 8b5b5bf8b7871b85bd0fe5cd179f16693e4fbb2a
Summary: The binhunk is for binary data, therefore we need to use byte strings.
Reviewed By: xavierd
Differential Revision: D20290069
fbshipit-source-id: 9cd763b76df389a1f7b65aecf0be4aa36a85cf91
Summary:
On python3, no need to replace stdin/stdout with a bytes based one, keep them
as is.
Reviewed By: xavierd
Differential Revision: D20290068
fbshipit-source-id: b10b69185a52c79e7c67f69f6a8a96937c64a550
Summary:
Not sure if this is the proper solution, but the template engine refuse to work
with anything but utf8 strings, but the diff operation only work on bytes.
Let's convert the return of diff to utf-8.
Reviewed By: DurhamG
Differential Revision: D20927680
fbshipit-source-id: 25c2947cac417448ca3521c2d5478fa8eebef04f
Summary:
In Python3, the error returned from binascii.unhexlify changed, from a generic
TypeError to a binascii.Error. Therefore, wrap the binascii function and catch
the binascii.Error before raising a TypeError.
Reviewed By: DurhamG
Differential Revision: D20924129
fbshipit-source-id: 33f852ea97396af715ef73630e0dd1b4324eb707
Summary:
I can't say that I understand `mdiff.splitnewlines`. In my test it does not
behave the way it reads and I don't know why. The stripping that it's supposed
to do doesn't happen for some reason. It behaves like splitlines.
I believe that rcutil used '\n' for line termination because it was relying on
Python to do the conversion to system line end. I updated to use os.linesep
now that we encode the contents.
Reviewed By: quark-zju
Differential Revision: D20935377
fbshipit-source-id: 0958fdff03950ab0a4b2da02e4333b5438ac5c70
Summary:
Bytes and Str usages mostly. __iter__ seems to have been incorrectly
converted to python 3 previously.
Reviewed By: xavierd
Differential Revision: D20933166
fbshipit-source-id: 10e63e90bd83c70a51dd808e9b5073ab8d766e71
Summary: We should read/write to it via as utf8.
Reviewed By: DurhamG
Differential Revision: D20923404
fbshipit-source-id: 86cdc329395d60c88637f24d3c7c5caedcc7111a
Summary:
The computation of commit obsolescence is inconsistent. If we compute the full
set of obsolete commits in `mutation.obsoletecache.obsoletenodes`, then we
correctly ignore public commits as they cannot be obsolete.
However, if we compute the obsolescence state for a single public commit with
`mutation.obsoletecache.isobsolete`, and that commit somehow has a visible
successor, then we will incorrectly consider the commit as obsolete.
Similarly, `allpredecessors` and `allsuccessors` should stop when they hit a
public commit.
Reviewed By: quark-zju
Differential Revision: D20892778
fbshipit-source-id: 223cb8b2bc9f2f08124df6ff51c2eb208bb8eb5f
Summary: This makes the tracing features easier to use.
Reviewed By: DurhamG
Differential Revision: D19797703
fbshipit-source-id: fb5cb17cd389575cf0134a708bcd9df3b90e9ab4
Summary:
Somehow, enabling VT can fail when writing to the pager, but this doesn't
mean that the pager doesn't support VT mode, so let's just ignore the error
when the pager is active.
Reviewed By: DurhamG
Differential Revision: D20906374
fbshipit-source-id: 7cba52817bc8e4dc91d5d50e856ad8af7fc9542c
Summary:
Add a command to print visibleheads. This was part of my attempt to check if
visibleheads can accidentally include public commits and if there are a way
to remove it. I ended up thinking D20808884 might actualy solve the only case
that visibleheads include public heads.
I also tried to add strong verification so that the visibility layer never
writes public nodes. That's for non-narrow-heads use-cases. However, the
phasescache + repoview layer is kind of messy in a way that inside a
transaction there is only one "repo" that has the right in-memory, dirty
"phasescache" and other repos will load the (stale, wrong) phasescache from
disk. That means if we test phases in visibility before transaction flushes,
we won't be able to access the latest phases information correctly. So I
gave up this approach too.
Anyway, I wasn't able to add a new interesting test, but the utility built for
the test seems useful. Therefore this change.
Reviewed By: sfilipco
Differential Revision: D20829136
fbshipit-source-id: 5ebafefac820ebb4044db63b7892ffaa341c0573
Summary:
Make the error cleaner and more actionable. We don't autopull the commit
because the revset layer might be not ready for it (ex. it expects commit
graph to be immutable and might have done some calculations based on the
old graph already).
Reviewed By: sfilipco
Differential Revision: D20845159
fbshipit-source-id: c51f2f52c612ff14a88fb891c10d1faad1094635
Summary: This makes it easier to reason about.
Reviewed By: sfilipco
Differential Revision: D20845158
fbshipit-source-id: 4220c7790a2d3bb83380b65d3de469b42846b34d
Summary:
The old linear changelog search does not scale and does not work with segmented
changelog, which makes commit data lazy. The "forksearch" is also problematic as
it can produce non-deterministic results. Let's disable the linear search by
default with the intention to remove it if nobody complains.
Reviewed By: sfilipco
Differential Revision: D20845161
fbshipit-source-id: 6911035f146e15c88925217ee9940db59ea2d1c1
Summary:
The manifest node is hard for debugging. Attach the commit as the context and
print it out.
Reviewed By: sfilipco
Differential Revision: D20829139
fbshipit-source-id: ff65d902f56bc79c2d5f2c3ec9cf79a620fd70fc
Summary:
This allows providing context about errors, similar to Rust's "context"
features.
Reviewed By: sfilipco
Differential Revision: D20845160
fbshipit-source-id: 789a4c1b7356fd6eb4b1bcb71efd205d2d45e984
Summary: The template should use mutation instead of obsutil if mutation is enabled.
Reviewed By: markbt, simpkins
Differential Revision: D20901109
fbshipit-source-id: a2b587ddf2a03965886f753e54e075d5d9064f05
Summary:
Old is defined by being based on a commit that is more than 30 days old.
The build date is taken from the version string.
One observation is that if we fail to release in more than 30 days then all
users will start seeing this message without any way of turning it off. Doesn't
seem worth while to add a config for silencing it though.
Reviewed By: quark-zju
Differential Revision: D20825399
fbshipit-source-id: f97518031bbda5e2c49226f3df634c5b80651c5b
Summary:
When syncing, if the local copy of the remote bookmark is `None` then the
remote bookmark has been deleted, and shouldn't be added to the set of new
remote bookmarks.
Reviewed By: DurhamG
Differential Revision: D20855909
fbshipit-source-id: 84a7a78ec0ab179e4a14d946b37f496f3dbde03a
Summary:
When falling back to convert_revision to get the svnrev for a commit to return
it as a globalrev, it seems like it would make sense to verify that the
convert_revision is indeed a svnrev (and not, say, a git commit hash from
hggit).
Reviewed By: farnz
Differential Revision: D20891268
fbshipit-source-id: 2451ad787fcce7b10b6a405f2855313ca51f5b3e
Summary:
Similar to D19957251, failing to prefetch trees post-pull is not a fatal error.
This recently became an issue because with these conditions:
- We're migrating a repo from hg to Mononoke, and Mononoke has lagged
bookmarks, esp. remote/master.
- We're still using Mercurial to serve infinitepush pulls (ex. scratch
bookmarks). Mercurial will return bookmarks that are not synced to
Mononoke yet.
- I made a recent change (D20531121) that moves selectivepull logic that
changes `pull -r HASH` to `pull -r HASH -B master` to core. The CI
tier didn't have selectivepull enabled that will pull bookmarks, but
now they also pull bookmarks.
- The tree prefetch logic uses unpatched paths.default to fetch `master`,
which can be an unknown commit for Mononoke.
Ideally, the final state is to rely on Mononoke for everything. There is
only `paths.default` which points to Mononoke. For now, as a temporary
fix let's just make post-pull tree prefetch non-fatal. Once this happens,
it will be logged to the cloud table with fatal=false.
Reviewed By: krallin
Differential Revision: D20873599
fbshipit-source-id: 291a2dc41c7c602f004ad8df71e19a48a98e9c5e
Summary:
This makes it easier to spot typos.
The config sparse.missingwarning affects everytime a sparse profile is used.
In this case we just want to check at "enableprofile" time. So avoid using
that config option.
Reviewed By: DurhamG
Differential Revision: D20846313
fbshipit-source-id: 79f80d5e01abfe1633f2597074e9acb5cda60fec
Summary:
Tests are redirected, but we still want colors in them (to verify that ANSI
escape code works). Since VT mode will always fail, simply do not try to
enable it in this case.
Reviewed By: DurhamG
Differential Revision: D20875423
fbshipit-source-id: bd6b0046de9558d957e0df082d4392ad00e0b551
Summary:
Somehow the constant isatty and config reading shows up in my profile. Since
this is not intended to change, just cache it.
Reviewed By: DurhamG
Differential Revision: D20830511
fbshipit-source-id: de36c0a958f6a6046a05db0abf976994435b9e3c
Summary:
For our HgExternalSync jobs that pull from git, we don't really use most of the
bells and whistles of hggit. Notably, we don't care about bookmarks: we only
ever pull master, we never update to it, we only ever look at `-r tip`.
However, we do care about things that are actually much harder to fit in a
world where we try to pretend the remote git repository is actually a hg
repository we can pull from.
Notably, we'd like to enforce limits on how many commits we pull (and convert)
at a time, so that if we fall behind a little bit, we don't start falling even
more behind by having to convert bigger and bigger batches of commits. If we're
trying to pretend fetching from git and converting commits is actually a pull,
then that seems harder to pull off (we'd need to somehow rewind the remote head
we're pulling before importing it).
So, this adds a new external-sync command to hggit that basically the bare
minimum that we do need. It lets you specify a git remote and a head you care
about, and import up to N commits from it. That's it — no bookmarks are updated
or anything (but the git-mapfile is, of course). The only thing that changes is
your commits.
If you actually want to interact with your git repository on an ongoing basis
as if it were a remote hg repository, this is completely useless, but that
isn't what we actually do, so that should be OK.
As part of this, I've modified a few other parts of git_handler to remove
places where we called a `uri` `remote_name` (which is a bit confusing), and a
place where we were asking for a `remote_name` parameter that I don't have
here, but which we also didn't actually need (in `import_git_objects`).
Reviewed By: farnz
Differential Revision: D20836601
fbshipit-source-id: 96230e6e8269d0472404414948fd2f02aa98d79c
Summary:
Evolution crashes if you try to create obsmarkers in a repo where obsmarkers
are not enabled. Instead, check before reviving commits with obsmarkers that
it is enabled.
Reviewed By: krallin
Differential Revision: D20868173
fbshipit-source-id: b3a08ef05ddc13e9b78df8ac7c1ca1ef808b7602
Summary:
Let's mandate a 256 color terminal in Windows. We also update less to default
to using VT mode: https://github.com/gwsw/less/pull/29
With this, the colors appears to be working as expected on Windows!
Reviewed By: quark-zju
Differential Revision: D20826922
fbshipit-source-id: 5d2a149e2698b3ccc7bf6388b5cd97da20898eb7
Summary:
For whatever reason, ncurses doesn't interpret the arrow keys properly... Use
whatever is returned from it instead.
Reviewed By: kulshrax
Differential Revision: D20821432
fbshipit-source-id: 019426a1333538a479fb3dd8d147fb9cb4ba6559
Summary:
Move the attribute so it won't cause an AttributeError when remotenames is not
enabled.
As we're here, also move it to repo._remotenames from repo. This makes the repo
object a bit clearner and avoids loading the accessed bookmarks if remotenames
are not accessed at all.
Reviewed By: singhsrb
Differential Revision: D20821070
fbshipit-source-id: b5a01016451ee202f778cadb2dd418400406fa6b
Summary:
We've seen a number of recent cases where hg stops using watchman for
status because state-enter or state-leave fail because multiple enters or leaves
are called out of order. The root cause needs to be found, but in the mean time
we should not allow such errors to block of from using watchman entirely.
Let's eat these gracefully and just log that the issue happened.
Reviewed By: quark-zju
Differential Revision: D20807004
fbshipit-source-id: 517d9562febc226ce7db836435ca2ca9db1c68fb
Summary:
On shared devservers, all users in svnusers should be able to read/write onto
the stores, set the umask to properly reflect this.
Reviewed By: ikostia
Differential Revision: D20817266
fbshipit-source-id: 8a865f499a35313afd3cbeb5f989175862a3549f
Summary:
This updates fastheaddiscovery to use an unfiltered repository when it attempts
to check if the nodes the server sent are present locally.
Reviewed By: quark-zju
Differential Revision: D20792006
fbshipit-source-id: 14ba9605d79ba54f3f4143d6d8ec65357e3d8c07
Summary:
The new API does nothing that cloud sync does not want: bookmarks, obsmarkers,
prefetch, etc. Wrappers to disable features are removed.
This solves a "lagged master" issue where selectivepull adds `-B master` to
pull extra commits but cloud sync cannot hide them without narrow-heads. Now
cloud sync just does not pull the extra commits.
Reviewed By: sfilipco
Differential Revision: D20808884
fbshipit-source-id: 0e60d96f6bbb9d4ce02c04e8851fc6bda442c764
Summary: This makes the upcoming change easier to verify.
Reviewed By: sfilipco
Differential Revision: D20808883
fbshipit-source-id: 5563601125ef5c961785f7275fd82fc3fefe53ff
Summary:
We have seen one report that the race condition caused lagged remote/master.
The race condition is:
- pull everything: get all commits, but not remotenames.
- expull: get all remotenames. Some point to new commits received by the
server that are unknown to the client. The client ignores them.
This diff adds a double check to pull again if it sees remotenames pointing
to unknown nodes.
This won't be necessary if we migrate everyone to selectivepull, and clean up
the pull command to use the race-free repo.pull API. Because the new code path
will first query the bookmarks, then pull the heads explicitly.
Reviewed By: DurhamG
Differential Revision: D20703268
fbshipit-source-id: 42bd0201c3c4f709f25ad0fea3a191db9991e4fc
Summary:
This makes it possible to do things like `hg up remote/foo` without having to
`hg pull -B foo` first.
Reviewed By: DurhamG
Differential Revision: D20531122
fbshipit-source-id: e95b2f0a49e4b815c136450d1f352a7973cb72ed
Summary:
The new `repo.pull` API aims to be tech-debt free. It does:
- Pull nothing instead of everything if nothing is specified.
- Update remote names that are explicitly specified, not everything the server
has.
- Do not update local bookmarks.
The direct motivation is to implement autopull `remote/foo` behavior while not relying on the pull command.
Reviewed By: DurhamG
Differential Revision: D20531119
fbshipit-source-id: 80c3bdd5556126d81af099a74f1345ecc94904b7
Summary: This will be used by the next diff.
Reviewed By: DurhamG
Differential Revision: D20661056
fbshipit-source-id: 3b4793bc0737bd64196da5b99c2d4da6f5e48efb
Summary:
During the roll-out of commitcloud remotebookmarks sync, some clients may have it enabled, and others may have it disabled.
This can result in some clients having an empty set of remotebookmarks in their sync state for a version which actually
does have remotenames in the cloud workspace.
This becomes a problem when those clients start syncing remotebookmarks, for two reasons:
* They are unable to correctly apply the new cloud remotebookmarks, as they do not have the previous state with which to break ties, and the tie-breaking code doesn't work correctly if the old cloud remotebookmark is `None`.
* When they try to send the remotebookmarks delta to the server, they don't have the old state, and so compute the delta incorrectly. This results in `Duplicate entry for key` errors in the service.
We can handle this as follows:
* When syncing remote bookmarks from the cloud, ensure we correctly apply the remote bookmark updates, even if the old cloud remotebookmark is `None`.
* If the local repo is up-to-date with respect to the workspace, but the last sync state had no remote bookmarks recorded, sync the remote bookmarks again. This may involve pulling some more commits if the cloud remote bookmarks are ahead of the local remote bookmarks. If there truly were no remote bookmarks then this will be a no-op.
Reviewed By: quark-zju
Differential Revision: D20792360
fbshipit-source-id: cd1ed371d8d5b1be2767c8a8d4836ea870a4467b
Summary: Add a hint that points to customization and troubleshooting guides for the new graph renderer.
Reviewed By: xavierd
Differential Revision: D20763338
fbshipit-source-id: ee6d2464ae5955f0f0bf52d1994adfa2b74b3367
Summary:
This updates hg rage to disable colors while producing the hg rage. This makes
the pastes a little more readable since they won't contain escape codes.
Reviewed By: farnz
Differential Revision: D20738945
fbshipit-source-id: e323834beb6458083642e34a40afa3d896288b42
Summary:
Instead, it's present in the fileslog. Use that instead.
This is the error I just got while doing a `hg top`:
error updating fbcode/eden/scm/lib/revisionstore/src/contentstore.rs: facebook::eden::HgImportPyError: AttributeError: 'bindings.revisionstore.contentstore' object has no attribute 'commitpending'
Reviewed By: quark-zju
Differential Revision: D20715560
fbshipit-source-id: 59c4b62ae8d0f182824e126e68b174c0ef3cdba0
Summary:
Similar to the previous diff. Sometimes the narrow-heads config is on but the
dependencies are off (ex. remotenames, visibility). In that case narrow-heads
should be considered off. The repo init function takes care of that but code
paths before repo init needs special care.
Reviewed By: simpkins
Differential Revision: D20699684
fbshipit-source-id: befac335bde0be6b8a42a5a9f7cee1edbbef3c55
Summary:
Some tests fail because `hg init` sets the wrong initial store requirements and
then perform a narrow-head migration down, which prints extra messages. Fix them
by making sure the initial store requirements are the same as what the migration
code path expects.
Reviewed By: simpkins
Differential Revision: D20698637
fbshipit-source-id: 1422a4ea78222617d0e3f9631ad883d5a3fe6bb7
Summary:
One of the main drawback of the current version of repack is that it writes
back the data to a packfile, making it hard to change file format. Currently, 2
file format changes are ongoing: moving away from packfiles entirely, and
moving from having LFS pointers stored in the packfiles, to a separate storage.
While an ad-hoc solution could be designed for this purpose, repack can
fullfill this goal easily by simply writing to the ContentStore, the
configuration of the ContentStore will then decide where this data will
be written into.
The main drawback of this code is the unfortunate added duplication of code.
I'm sure there is a way to avoid it by having new traits, I decided against it
for now from a code readability point of view.
Reviewed By: DurhamG
Differential Revision: D20567118
fbshipit-source-id: d67282dae31db93739e50f8cc64f9ecce92d2d30
Summary: There is no need to pass the path twice, once is enough.
Reviewed By: DurhamG
Differential Revision: D20567117
fbshipit-source-id: 169a088aa7558b4c25f8fbdecfe59bdd0d7575ef
Summary:
This enforces certain selective pull logic in core. Namely, rewrite `pull -r X`
to `pull -r X -B master`.
Unlike selectivepull in remotenames, `pull` (pulls everything) won't be
rewritten to `pull -B master` (which pulls less commits and names).
Therefore this change always adds more commits to pull, and therefore should not
break existing users. Eventually we want the "not pulling everything" behavior,
but right now this just fixes `pull -r X` to also update important remote names.
Reviewed By: markbt
Differential Revision: D20531121
fbshipit-source-id: af457b5ddb1265b61956eb2ee6afb7b7208293e0
Summary: Now we have functions to get selectivepull names in core. Use that instead.
Reviewed By: markbt
Differential Revision: D20531118
fbshipit-source-id: a0c20c491baf1b0ad71e80f870703bb4b983f19c
Summary: This makes it possible for core to always pull related bookmarks.
Reviewed By: markbt
Differential Revision: D20531120
fbshipit-source-id: 52f0834b517e03567e240f57616370d79a227abe
Summary:
This makes it possible to use template like `{remotenames}` or revset like
`remotenames()` without enabling the remotename extension.
Rarely used revsets like `upstream()` and `pushed()` are not moved.
Reviewed By: markbt
Differential Revision: D20529360
fbshipit-source-id: ea95b3324f974e112909cdd79ce662940a4f9b7c
Summary:
This makes it possible to resolve remotenames without enabling the remotenames
extension.
The config check `if repo.ui.configbool("remotenames", "bookmarks")` is dropped
intentionally as we only use remotenames for bookmarks, not named branches.
Since this only enables resolving more names, without disabling or changing
other features, and the remotename namespace priority is lower than local
bookmarks (ex. if a local `master` exists, then `master` will be using the
local bookmark, not the hoisted remote name), it should not cause breaking
changes.
Reviewed By: markbt
Differential Revision: D20529359
fbshipit-source-id: 4126faee1bb7f43ba547fab05dd6197b2e65c1fc
Summary:
This moves part of remotenames that provides the `repo._remotenames` object to
core. It should not change behaviors but merely makes `repo._remotenames`
available in core.
Reviewed By: markbt
Differential Revision: D20529358
fbshipit-source-id: 11c8538a0190101b09a4cb082018e73643a257e2
Summary:
While failures in the Rust updater aren't expected, at least one valid case
requires requires retrying the operation in Python: old-style LFS pointers.
When these are stored in packfiles/indexedlog, only the Python code knows how
to deal with them, and thus the operation needs to be retried there.
Reviewed By: DurhamG
Differential Revision: D20603709
fbshipit-source-id: 7d24ba573f0ff540906d909f1b4440fd4d3469a6
Summary:
When we deserialize commit extras they come out as a str->str dict, so
let's make sure clients add values that are also strings. We do this by type
checking a serialization time. This fixes an issue where D19942522 assumed
extras were strings and hit a type error for globalrevs, which were ints.
Apparently the code that reads globalrevs from the extras never treats it as an
int anyway, so none of the reading code needed to be updated to convert the
string back to an int.
Reviewed By: singhsrb
Differential Revision: D20631889
fbshipit-source-id: 8c8b3c9a9f3369376e08146d670f2d6321df141f
Summary:
If `HGPLAIN` is set, and the non-legacy graph renderer is in use, use the ASCII
renderer to ensure compatible output on all platforms and configurations.
Reviewed By: farnz
Differential Revision: D20622425
fbshipit-source-id: b25a8d0526652bab07059492f7adbb684b5cbee7
Summary:
The markforrefresh only exist in the legacy Python contentstore, commiting the
current store will have the same effect for the Rust store.
Reviewed By: DurhamG
Differential Revision: D20562032
fbshipit-source-id: de12cfd65d992395c9b9894d1f05cf50d86b0191
Summary:
In a strongly typed langage, using strings should be avoided whenever possible
as they do not provide the safety guarantees that types provide.
I took the liberty of removing all the filesystems that are not relevant for
Mercurial for simplification reasons. If needs arise, we can always add a new
FsType to the enum.
Reviewed By: DurhamG
Differential Revision: D20517138
fbshipit-source-id: 0a38b53c6a87f05f4b2d664038e10c4293de96ae
Summary:
This makes it so `hg status A` won't complain about race conditions about file
`B`. As we're here, the detector logic was also moved to the filesystem layer.
Reviewed By: DurhamG
Differential Revision: D20568151
fbshipit-source-id: 713e3775a14674fd170df391977d42d9dee0f718
Summary:
Originally, we wanted to log EdenAPI batch sizes to Scuba in order to A/B test the effect of batch size on fetching speed. It turns out that the original arbitrary choice of 1000 works pretty well, so there isn't really a need to log this to Scuba anymore.
In addition to removing these fields, this diff moves the "http_enabled" column to be logged from the config by the wrapper instead of being manually logged by hg.
Reviewed By: quark-zju
Differential Revision: D20568104
fbshipit-source-id: 1e113489d005e93e6283f7e4a439b1c78b6f0fb9
Summary:
Fix code in `memcommit/serialization.py` that was assigning a `str` to a
`bytes` variable.
Reviewed By: singhsrb
Differential Revision: D20564418
fbshipit-source-id: 578ebe9d2b8823353d5a4727f6ea888678ca881f
Summary:
There isn't really much to annotate here, but this lets us eliminate a couple
`pyre-fixme` comments about not being able to find these modules during type
checkign.
Reviewed By: singhsrb
Differential Revision: D20550267
fbshipit-source-id: 271f8406890787c0613294a9047365fdebcdeda1
Summary:
These were copied into the `edenscmnative` directory in D15798642, but never
removed from their original location.
Reviewed By: singhsrb
Differential Revision: D20550268
fbshipit-source-id: dadce279f1717a66ece86376bd51179b8669cc8a
Summary: For LFS blobs, these can be obtained very easily by querying the ContentStore.
Reviewed By: DurhamG
Differential Revision: D20504235
fbshipit-source-id: 937ef20184d6524b1355565f9ab81e40b56d7ab0
Summary:
This is the start of migrating blackbox events to tracing events. The
motivation is to have a single data source for log processing (for simplicity)
and the tracing data seems a better fit, since it can represent a tree of
spans, instead of just a flat list. Eventually blackbox might be mostly
a wrapper for tracing data, with some minimal support for logging some indexed
events.
Reviewed By: DurhamG
Differential Revision: D19797710
fbshipit-source-id: 034f17fb5552242b60e759559a202fd26061f1f1
Summary:
The `all()` revset is much slower with narrow-heads for correctness. Use an
alternative that is fast.
Reviewed By: markbt
Differential Revision: D20528063
fbshipit-source-id: c8ae35e67e60407406ca81d67878278392626e9a
Summary: Based on comments on D20382825, we need to make sure that `_gettrees()` knows for sure whether on-demand tree fetching is in use in order to properly identify missing nodes in the response.
Reviewed By: quark-zju
Differential Revision: D20520439
fbshipit-source-id: ffa6d62dbe8b6f641b1dacebcb6f94ceae714c1b
Summary:
This allows structures like BTreeMap to own and store Segment.
It was not possible until D19818714, which adds minibytes::Bytes interface for
indexedlog.
In theory this hurts performance a little bit. But the perf difference does not
seem visible by `cargo bench --bench dag_ops`:
# before
building segments 714.420 ms
ancestors 54.045 ms
children 490.386 ms
common_ancestors (spans) 2.579 s
descendants (small subset) 406.374 ms
gca_one (2 ids) 161.260 ms
gca_one (spans) 2.731 s
gca_all (2 ids) 287.857 ms
gca_all (spans) 2.799 s
heads 234.130 ms
heads_ancestors 39.383 ms
is_ancestor 113.847 ms
parents 251.604 ms
parent_ids 11.412 ms
range (2 ids) 117.037 ms
range (spans) 241.156 ms
roots 507.328 ms
# after
building segments 750.129 ms
ancestors 53.341 ms
children 515.607 ms
common_ancestors (spans) 2.664 s
descendants (small subset) 411.556 ms
gca_one (2 ids) 164.466 ms
gca_one (spans) 2.701 s
gca_all (2 ids) 290.516 ms
gca_all (spans) 2.801 s
heads 240.548 ms
heads_ancestors 39.625 ms
is_ancestor 115.735 ms
parents 239.353 ms
parent_ids 11.172 ms
range (2 ids) 115.483 ms
range (spans) 235.694 ms
roots 506.861 ms
Reviewed By: sfilipco
Differential Revision: D20505201
fbshipit-source-id: c34d48f0216fc5b20a1d348a75ace89ace7c080b
Summary:
EdenFS is reported as `osxfuse_eden` on OSX after D20313385.
Update the fscap table to avoid slow paths testing fs capabilities.
Without this diff, churns on edenfs OSX will trigger undesirable watchman
events.
Reported by: fanzeyi
Reviewed By: fanzeyi
Differential Revision: D20518902
fbshipit-source-id: 2e8e472df16d08b17834b2c966c065bbaad052fe
Summary:
We see some hgbuild jobs failing because the order of errors is
different from what I see on my devserver. Let's sort them to make them stable.
This is presumably because we're operating in the order returned by readdir,
which is not guaranteed to be sorted.
Reviewed By: xavierd
Differential Revision: D20500566
fbshipit-source-id: bd4d3db1b77cd4bd7259f9bcc10bc65649fae7c6
Summary: We don't really need the Rust workers for this, as we do not expect thousands of files to be changed during an in-memory merge.
Reviewed By: DurhamG
Differential Revision: D20495141
fbshipit-source-id: e72f8c4b01deee46ee72364dcd6716692c4103ab
Summary:
python2 doesn't exist on centos 8, and rather than guess
at which interpreter to use, run using the built in interpreter
from our rust binary which is accessible via `sys.executable`
under the `debugpython` subcommand.
Reviewed By: quark-zju
Differential Revision: D20379035
fbshipit-source-id: 57863427241ef01e5c0e8debbb5baf056fd41d65
Summary:
Remove the custom `GzipFileWithTime` class from `mercurial/archival.py`
This code was added in 2007. Presumably back then Python's standard
`gzip.GzipFile` class did not support the `mtime` argument. However, this
argument is present in Python 2.7+, and we don't care about older versions.
The custom `GzipFileWithTime` class breaks in Python 3.8 since Python 3.8
added an extra `compresslevel` argument to the internal `_write_gzip_header()`
method.
Reviewed By: pixelb
Differential Revision: D20484845
fbshipit-source-id: 4e381799d8537c97cd1993273c8efd02743531df
Summary: Let's enable it for tests. We'll slow roll it in production.
Reviewed By: quark-zju
Differential Revision: D19543790
fbshipit-source-id: be7d18dd8ffe51615a27c39ebf4247ec405b4097
Summary:
The mutation store stores entries with a floating-point timestamp. This
pattern was copied from obsmarkers.
However, Mercurial uses integer timestamps in the commit metadata (the
parser supports floats for historical reasons, but only stores integer
timestamps). Mononoke also uses integer timestamps in its `DateTime`
type.
To keep things simple, switch to using integer timestamps for mutation
entries. Existing entries with floating point timestamps are truncated.
Add a new entry format version that encodes the timestamp as an integer.
For now, continue to generate the old version so that old clients can
read entries created by new clients.
Reviewed By: quark-zju
Differential Revision: D20444366
fbshipit-source-id: 4d6d9851aacb314abea19b87c9d0130c47fdf512
Summary:
Tracking the origin of mutation entries did not prove useful, and just creates
an un-necessary overhead. Remove the tracking and repurpose the field as a
version field.
Reviewed By: quark-zju
Differential Revision: D20444365
fbshipit-source-id: 65ff11ee8cfe77d5e67a83d03a510541d58ef69b
Summary:
We want to delete all the non-treestate dirstate implementations. Let's
start throwing an exception if treestate is not enabled. We temporarily have a
bypass in case we break an important usecase in the process.
This also sets the standard new repo to be created in treestate mode, but adding
treestate to newreporequirements.
This was landed once as D19204621 but was backed out because eden backing repos
were using the odl formats and hadn't been upgraded. We fixed that, and now the
data shows ~10 people still using repos in this condition
(https://fburl.com/scuba/dev_command_timers/zxb5hsg2). Some of them are broken
repos, some are ancient eden repos and a simple eden rm and eden clone should
fix them, some are simply old non-eden repos that no one has run commands in in a while.
Reviewed By: xavierd
Differential Revision: D20472234
fbshipit-source-id: 509b4f22b6ac4741b205ef69decfb26e56aebaf8
Summary:
Together with `debugmetalogroots`, this allows some kind of "time travel" to
investigate repo states in the past.
Reviewed By: sfilipco
Differential Revision: D20449409
fbshipit-source-id: ed5c134f9e9ee235b24f45c1aa35867a55a71fe5
Summary:
This makes the content of `bookmarks` stable.
It will make the metalog root IDs stable in tests.
Reviewed By: sfilipco
Differential Revision: D20449410
fbshipit-source-id: 969be63ab231f5865ec62e99398b5318d4257093
Summary:
Purge needs to be able to see what directories the walker traversed, so
it can delete them if they are empty. Instead of having the walker call
match.traversedir (which it seems like a bizarre pattern to use the matcher as a
holder for a non-matching related function), let's have the walker return an
enum and have an option to return directories.
At the python layer we then translate this into match.traversedir calls, but we
can clean that up later.
Reviewed By: quark-zju
Differential Revision: D19543795
fbshipit-source-id: cc51c86c91799d3df2c65d25a7b6cfe810206d0a
Summary:
On case insensitive systems we need to normalize file case. I've made a
rust case normalizer, but it requires some more tweaks. In the mean time, let's
handle this at the matching and output stages of the rust walk.
This is probably the pattern we want to follow later anyway, so the walk is
completely decoupled from normalization.
Reviewed By: kulshrax
Differential Revision: D19543797
fbshipit-source-id: 2ef8bdcecb2611a08680441fc030c64c2f4097d1
Summary:
The flat git map file can be huge and writing to it can hang for many
seconds or minutes on loaded systems since it calls fsync. Since we actually
rely on the indexedlog map file instead of the flat map, let's just stop writing
the flat map when indexedlog is enabled. This was previously done to enable
falling back, but we've had no issues and never needed to fallback.
Reviewed By: quark-zju
Differential Revision: D20443392
fbshipit-source-id: a3e7dbdb2b37e2e99df774f540b05d06414fb279
Summary:
Since configparser enforces utf-8 config files (because pest wants Rust strings),
let's migrate from Bytes to Text to remove extra encoding conversions.
Previously this was blocked by the lack of ref-counted text (since the "source"
of each config location is the entire config file). Now minibytes provides Text
so we can use it.
This unfortunately requires dependent code to be updated. The pyconfigparser
interface is in theory wrong - it shouldn't return utf-8 bytes but
local-encoded bytes. I think it's cleaner to make pyconfigparser unaware of
HGENCODING, so I changed pyconfigparser to use unicode, and add compatibility
layer in uiconfig.py.
This also fixes non-ascii encoding issues on user name (especially on Windows).
The hgrc config file should be in utf-8 and the config parser returns explicit
unicode types, and Python code round-trip them with local encodings.
Reviewed By: markbt
Differential Revision: D20432938
fbshipit-source-id: b1359429b8f1c133ab2d6b2deea6048377dfeca1
Summary:
For users on slow connections, hg debugnetwork may not complete within 20s.
Reduce the amount of data we use for the download and upload tests, so that we
have a better chance for the tests completing within the 20s limit.
Reviewed By: farnz
Differential Revision: D20439271
fbshipit-source-id: b3cb0d7e96673c8052022c7606cca80db1788370
Summary: At current, there are some specific AOSP branches that are causing problems for the importer. It would be helpful to be able to run convert jobs that only run on a subset of branches for testing. This commit adds a configuration setting for a whitelist that allows us to specify which repo branches we want to convert, and we only convert those branches.
Reviewed By: tchebb
Differential Revision: D18534728
fbshipit-source-id: bf59605d9c6026e26914c4a7cbe9493bdee77ee5
Summary:
The metalog message is just for display purpose so it does not have to be
byte-to-byte accurate. This solves potential crashes on Python 2 Windows.
File "c:\Tools\hg\python27.zip\edenscm\mercurial\transaction.py", line 587, in _writemetalog
" ".join(map(util.shellquote, pycompat.sysargv[1:])), int(util.timer())
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 1: invalid utf-8
The logic will become more "correct" when we migrate to Python 3.
Reported by: markbt
Reviewed By: markbt
Differential Revision: D20422747
fbshipit-source-id: 41123d132a1e545db77d7321099da611668174f4
Summary:
hg makes sure commit messages are utf-8. So there is no need to check it.
The check logic is also incorrect if the local encoding is not utf-8:
`ctx.description()` returns local-encoding-encoded bytes, which should not
be treated as utf-8 bytes.
Reviewed By: markbt
Differential Revision: D20424241
fbshipit-source-id: 05480e37335c3f2c7971c28c2e23c115dfc6fba7
Summary: This makes it possible to hide draft branches using `hg hide`.
Reviewed By: markbt
Differential Revision: D20403505
fbshipit-source-id: d316e02c24ce636fdc6593f95a5d974b1ba0fc85
Summary:
During cloud sync, the `_applycloudchanges` step will pull all the new commits
from commit cloud, and then record that it has updated to the new workspace
version in the cloud sync state.
If the transaction subsequently fails and gets rolled back, we will roll back
the pull (so the new commits vanish) but we will not roll back the recording of
having updated to the new cloud workspace version.
On the *next* cloud sync, commit cloud will think the user has deliberately
hidden all of those missing commits, and remove them from the cloud workspace.
Write the cloud sync state within the transaction, so if the transaction is
rolled back, we also roll back the recording of the new workspace version.
Reviewed By: quark-zju
Differential Revision: D20420240
fbshipit-source-id: f3d201f299e491cbff47fc6b703a4e1af4015b18
Summary:
Commit cloud sync works at the store granularity, so the status should be
stored in the store vfs. Move it there.
Improve the output of `hg cloud status` by logging more information about the workspace.
Reviewed By: quark-zju
Differential Revision: D20419223
fbshipit-source-id: 4f0d6b9aab55d2bbeaf89b489606f0bc25400de5
Summary:
This diff adds client-side support for the new `designatednodes` mode of the `gettreepack` wireproto command (implemented in D20307442), which allows fetching an exact set of specified tree nodes. This enables the performance benefits of BFS tree fetching over SSH (whereas this sort of fetching was previously only possible over HTTPS via EdenAPI).
To allow us to smoothly control the rollout of this feature, it is gated behind the `treemanifest.ondemandfetch` config option. In theory, this should not be needed since the client will check that the server supports `designatednodes` before attempting to do on-demand fetching, but if users report issues it's nice to have a killswitch.
Reviewed By: quark-zju
Differential Revision: D20382825
fbshipit-source-id: f9aef3fdae08ae84a9720deead3549fe2fba0b9b
Summary:
We cannot differentiate an empty list from a list containing the empty path only. This fixes that.
See D20309700 earlier in this stack for the server-side bits.
Reviewed By: quark-zju
Differential Revision: D20309699
fbshipit-source-id: 02efc747c3c5e2774f949ba0a6370169b55f3cf3
Summary:
In order to support gradual rollout of infinitepush for other backends (e.g. Mononoke), we need the ability to route read vs write requests separately. To achieve this, we need a separate path type: `infinitepush-write` (kind of like `default-push`, the naming inconsistency is a little unfortunate, as I don't want to use `infinitepush-push`).
The desired behavior of the new path type is as follows:
- takes precedence over `infinitepush` path when the user does `hg push -r . --to scratch/bla`
- replaces `infinitepush` path when the user does `hg push infinitepush -r . --to scratch/bla`
- absence of this path means draft pushes will go to `infinitepush` path
- draft pulls always go to `infinitepush` path, and *there's no fallback to `infinitepush-write`*
- commit cloud always talks to `infinitepush-write`, if it is present (meaning that commit cloud pulls do go to `infinitepush-write` path
- this is done, as commitcloud uses infinitepush paths to also check whether something is backed up
- and also, commitcloud may need to pull very soon after something has been pushed
Reviewed By: quark-zju
Differential Revision: D20368158
fbshipit-source-id: 59db174cebbf2b48765dff37bc93aad176c2d7c1
Summary: Those functions are reused in in a future diff.
Reviewed By: sfilipco
Differential Revision: D20367838
fbshipit-source-id: 944babf8c02f0560f8ac8ca5d7c4263432032715
Summary:
The `sparse` extension's dirstate tracking code attempts to read dirstate data
after the command has executed. For commands that didn't require accessing
the dirstate, this can end up reading it for the first time, and can this fail
in some circumstances. Ignore all errors that occur when trying to compute
this telemetry, so they don't break the main command functionality.
In particular, we see reproducible scenarios where `hg` crashes due to this
error when invoked by watchman to compute the list of files changed between
two commits. Watchman currently always invokes Mercurial with `HG_PENDING`
enabled, and this can cause dirstate loading to fail when there is pending
data.
Reviewed By: quark-zju
Differential Revision: D20402221
fbshipit-source-id: 782d9b6eff26d50ef20f080c0cbcbc852e425146
Summary: Remove the old Python, C implementation of getfstype.
Reviewed By: xavierd
Differential Revision: D20313385
fbshipit-source-id: 475c73343aae2fa2f5ad898c7b0879bfa2c80e93
Summary:
A common cause for automigration failure is if the repository has an abandoned
transaction. Just ignore the error and skip migration if this happens.
Otherwise important read-only commands (like `hg debugedenimporthelper`)
refuse to even start in this situation.
Reviewed By: farnz
Differential Revision: D20394546
fbshipit-source-id: abb75fe455e9ee815032431705df7f38ee50283a
Summary:
The code to disable visibility was writing to files without a transaction,
which causes a `ProgrammingError` to be thrown when using metalog.
Reviewed By: farnz
Differential Revision: D20394547
fbshipit-source-id: c529de84ed7b127df18e7f60572cece2169dc520
Summary:
Print output using `repo.ui.write()` rather than directly using `print()` or
`sys.stdout`. This is needed to make the output visible during the
Mercurial `.t` tests.
Reviewed By: farnz
Differential Revision: D20394549
fbshipit-source-id: 7ee50fe26e98c1c34b74ce054d2187050c42b2f3
Summary:
Add type annotations for the `edenscmnative.parsers` methods that handle
serializing and deserializing the dirstate data.
Reviewed By: quark-zju
Differential Revision: D19958218
fbshipit-source-id: 6e20efbc1b0a6ba15b297e47a1e6eec8ed47ee52
Summary:
If `ui.quiet` is set, debugnetwork still ends up outputting the output from the
remote `hostname` command. Prevent this from happening by redirecting the
output to a buffer and logging it separately through `ui.status`.
Reviewed By: quark-zju
Differential Revision: D20384651
fbshipit-source-id: 375687a8e75187a27ef30b441b2ac3a5c0823d00
Summary:
Allow selective running of a subset of the tests by passing options.
* `--connection` runs only the tests for determining connectivity
* `--speed` runs only the speed tests.
Not specifying any option runs all tests.
Reviewed By: quark-zju
Differential Revision: D20384652
fbshipit-source-id: 197e07809bec28ab2ce3fa73cfef6be0fb0fd5b7
Summary:
This command can be used to diff two trees and only print changed paths,
without changed file contents.
With a customized template it can also print changed flags, which can be useful
for watchman use-cases.
Reviewed By: sfilipco
Differential Revision: D20377436
fbshipit-source-id: dad79f1b891182fa612c446114f9daceb4ec5881
Summary: Those experimental revsets are not used in the past month. Remove them.
Reviewed By: sfilipco
Differential Revision: D20355671
fbshipit-source-id: 22d6053af01a56f23b7227b86ebe271aa2b42f41
Summary:
We currently log the server we talk to, which is very nice in order to
benchmark performance of Mononoke vs. hg.
Unfortunately, while this allows for breaking down the samples, it's a little
hard to identify the right samples to break down! Historically, we've
approximated this by looking at the client's hostname, but this isn't always
ideal.
Reviewed By: quark-zju
Differential Revision: D20369686
fbshipit-source-id: 785c67fde09e7b7fc4c30121d14f0bc26e92a355
Summary:
Symlinks are treated a bit differently from plain files, what is stored in the
ContentStore is the destination of the symlink, not it's content (well, the
content of a symlink really is it's destination).
For now, only unix platforms support symlinks, in reality this should be a
filesystem property as writing to ntfs-3g should have the same behavior as on
Windows.
For executable, we just need to mark the file as executable after writing to
it.
Reviewed By: quark-zju
Differential Revision: D20250943
fbshipit-source-id: 022dabc750125df32953a151df7da60db69b2cec
Summary:
During `hg update`, Mercurial forks multiple processes to write files on disk
concurrently, this is done as fetching blobs from the content store, and
writing them to disk is CPU bound. Usually, threads would be the preferred way
of speeding up such process, but unfortunately, Python has GIL that severely
limit the available concurrency. So, multiple processes were chosen.
Unfortunately, the multi-process solution also brings a lot of other issues,
more recently, we've had cases where the connections to the server and memcache
had to be dropped after the fork. In some other cases, this caused deadlocks.
And the solution is not effective on Windows.
Now that Mercurial is getting more and more Rust, we could instead go back to
the threads solution by using them in Rust, and have Python just push work to
them, this is exactly what this change does.
Things that are left to be done, but I wanted to get a diff out first:
- no file path audit
- no file backup
- no symlink creation
- probably other things I'm missing
Reviewed By: quark-zju
Differential Revision: D20102888
fbshipit-source-id: d47829fd7818b97710586b9851880f178048e27b
Summary:
It might be useful to know how long the end-to-end wireproto command to find
the master bookmark took, so log that.
Reviewed By: quark-zju
Differential Revision: D20344686
fbshipit-source-id: 8459a378a0c828a929eee5f9ffd34875a5b8d3cc
Summary:
For high latency connections we will need to warm up the connection before
starting the download or upload test. Facilitate this by making it possible to
run multiple tests within the same connection.
This changes the protocol for the speed-test command to be line-oriented commands.
Reviewed By: farnz
Differential Revision: D20344687
fbshipit-source-id: 1d2a815736caf974ed1aaaaf365b86343d41fe02
Summary:
On Python 2 the Mercurial JSON can be binary, which can break the telemetry
logger. Ensure the JSON string is valid utf-8 (although it might still be
invalid JSON).
Reviewed By: xavierd
Differential Revision: D20343845
fbshipit-source-id: 61e99e742bddf23c7fd5354a5754d79a0a452c28
Summary:
Add `hg debugnetwork` to the things that `hg rage` collects.
Some of the output from `hg debugnetwork` comes from the peer connection, which is not captured when using `ui.pushbuffer`, as it is written using the remote ui object created when the peer connection is set up.
To handle this, add `ui._outputui` as a way to redirect ui output to another ui object, and use this to redirect the output from the remoteui to the local ui object which is buffering output for hg rage.
Reviewed By: quark-zju
Differential Revision: D20307725
fbshipit-source-id: 3b79a77a39c6e2c5f8a7d5cc271ec466653d4db3
Summary:
Add a command that performs various network tests to check the connection to
the Mercurial server for SSH peers.
These tests are:
* Check we can look up the server name in DNS.
* Check we can make a TCP connection to the SSH server.
* Check we can connect and authenticate over SSH and run `hostname`.
* Check we can make a Mercurial wireproto connection and find the `master` bookmark.
* Check the download and upload speed to the server.
Checking the speed requires a companion speed test program on the server.
Reviewed By: mitrandir77, farnz
Differential Revision: D20305227
fbshipit-source-id: adba02a6a1c40ffc20d7bf9d962a5fcf85e06544
Summary: Make a bytestring with 'b""' to fix Python3
Reviewed By: DurhamG
Differential Revision: D20287313
fbshipit-source-id: 7455d1509684bfb0857a3b060bdcca7e658343fd
Summary: We need to encode/decode utf8 when reading/writing to the hgrc file.
Reviewed By: DurhamG
Differential Revision: D20286068
fbshipit-source-id: b1d6828fb62a83ad22414de6883004411f302142
Summary:
We saw >10 timeouts on Mononoke side from people who fetch a lot of files from corp network (https://fburl.com/scuba/mononoke_test_perf/qd15c5f1). They all tried to download a lot of files, and they 90 mins timeout limit.
Let's try to download these files in rather large chunks (e.g. 200000 files). That should help with resumability and prevent timeout errors, and make retries smaller.
It adds an overhead for establishing a new connection after every 200000 files. That shouldn't be a big problem, and even if it's a big problem we can just increase remotefilelog.prefetchchunksize.
Reviewed By: xavierd
Differential Revision: D20286499
fbshipit-source-id: 316c2e07a0856731629447627ae65c094c6eecc5
Summary:
On bad network link (such as on VPN), the reliability of the connection to
Mercurial might be fairly flaky. Instead of failing fetching files, let's retry
a bit first, in the hope that the connection will be back by then.
Reviewed By: quark-zju
Differential Revision: D20295255
fbshipit-source-id: d3038f5e4718b521ae4c3f2844f869a04ebb25e3
Summary:
We've had cases where a git commit goes in that shouldn't be translated
to Mercurial. Let's add an option to skip the commit. Instead of skipping it
entirely (which would require complicated logic to then parent the following
commit on the last converted commit), let's just convert the skipped commit as
an empty commit. This should cover the cases we've encountered so far.
Reviewed By: krallin
Differential Revision: D20261743
fbshipit-source-id: da401863b09c2ac727aae1ceef10a0e8d8f98a7e
Summary:
D18538145 introduced a transaction that spans the entire infintepush
pull. This has a couple of unfortunate consequences:
1. hg pull --rebase now aborts the entire pull if the rebase hits a conflict,
since it's unable to commit the transaction.
2. If tree prefetching fails, it aborts the entire pull as well.
Tests seem to work fine if we scope down this lock.
Reviewed By: xavierd
Differential Revision: D20260480
fbshipit-source-id: d84228ababdb5572401645f74e78df035bf1461b
Summary:
The `lines` renderer doesn't work if the output encoding doesn't support the
curved line drawing characters. In this case we should fall back to
`lines-square`.
Rename `lines` to `lines-curved`, and change `lines` to pick the best renderer
to use based on what is possible with the current output encoding.
Reviewed By: quark-zju
Differential Revision: D20248022
fbshipit-source-id: dfaf359426528a9cb515fb3e1d366fbfb15162ff
Summary:
The pager may accept a different encoding than either the process encoding or
the output encoding.
For example, on Windows:
* the process encoding may be cp1252 (which is used for all `...A` system calls.
* the output encoding may be cp436 (which is used for writing directly to the console).
* the pager encoding may be utf-8 (which is written to the console using more modern system calls).
To fix this, add a `pager.encoding` config option, which, when set, overrides
the output encoding when writing to the pager.
Reviewed By: quark-zju
Differential Revision: D20247650
fbshipit-source-id: 1e4d1246c95f2102763d879f9783d02acc193a73
Summary:
The change is in theory not necessary. However it improves the reliability on
OS crashes a bit, and can potentially workaround some bugs in filesystems
(as we saw in production where the atomic-written files are empty and the
system didn't crash).
The idea is, the `symlink` syscall does the file creation and "content" writing
together, while there is no way to create a file and write specific content
in one syscall. Note that the C symlink call uses 0-terminated string, and
the Rust stdlib exports it as accepting `Path`. To be safe, we encode binary
or non-utf8 content using `hex`.
For downgrade safety, the write path does not use symlink by default unless
format.use-symlink-atomic-write is set to true. This makes downgrade possible:
the read path is rolled out first, then we can turn on and off the write path.
The indexedlog Rust unit tests and test-doctor.t are migrated to use the new
symlink code paths.
Reviewed By: DurhamG
Differential Revision: D20153864
fbshipit-source-id: c31bd4287a8d29575180fbcf7227d2b04c4c1252
Summary:
When I run `hg doctor` in my www checkout it fails the assertion check of the
first line of visibleheads is "v1". Make it graceful so doctor can check and
fix other components.
Reviewed By: DurhamG
Differential Revision: D20147969
fbshipit-source-id: 6aee2cab962fcd0ef06a0611d288021e86621249
Summary:
Migrate away from some uses of revision numbers.
Some dead code in discovery.py is removed.
I also fixed some test issues when I run tests locally.
Reviewed By: sfilipco
Differential Revision: D20155399
fbshipit-source-id: bfdcb57f06374f9f27be51b0980652ef50a2c8e0
Summary:
`hiddenoverride` is a hacky implementation that preserves part of another hacky
`inhibit` extension. With our modern setup (inhibit or narrow-heads),
`hiddenoverride` is less useful. Therefore just remove it.
Reviewed By: sfilipco
Differential Revision: D20148011
fbshipit-source-id: f4a5f05b67ae6f315e9b07d50ef03018d6d05df5
Summary:
Blame can use a templater which doesn't support bytes. Let's just force
all blame output to be unicode, since it doesn't make a ton of sense to blame
binary files anyway.
Also fix test-annotate.py
Reviewed By: quark-zju
Differential Revision: D19907530
fbshipit-source-id: a7a47246368ed50f65486e824f93552872adc09a
Summary:
Notably, we drop all the encoding business when dealing with json
objects, and instead use mercurial.json.
Reviewed By: sfilipco
Differential Revision: D19888130
fbshipit-source-id: 2101c32833484c37ce4376a61220b1b0afeb175a
Summary: It was a list. Make it possible to use it as a string.
Reviewed By: xavierd
Differential Revision: D20144811
fbshipit-source-id: b280c0344215a4c23ab9c63d89f47adf34fb06f3
Summary:
The state entry of a dirstate tuple is a single character. In python 3
it's a unicode string. To parse it, previously we used 'C' which takes a single
character unicode string and (little did I know) returns an int. We were storing
this in a char, which causes corruption.
Let's switch to reading the string, and just grabbing the first byte.
Reviewed By: xavierd
Differential Revision: D20143094
fbshipit-source-id: d9946c0cefdafe0941f4bdac070659fac27f30e3
Summary:
Python json produces unicode strings in the parsed results. This breaks
when passed to parts of the code that now assert that byte strings are required
(like the wire protocol). Let's switch phabricator stuff to use Mercurial json,
which produces bytes in Python 2 and unicode in Python 3.
Reviewed By: ikostia
Differential Revision: D20123140
fbshipit-source-id: d1b11426736a0f43ff7e74acf709ab1fd70d5bfe
Summary: EdenFS is planning on throwing an error if a user requests a checkout while a checkout is already in progress. Often, this is already disallowed by a mercurial repository lock, but there are instances where these calls can still get through. We would like to disallow these calls to queue, so we will throw an `EdenError` instead. Without this handling, a full stack trace prints, so this just makes it a bit prettier for the user.
Reviewed By: simpkins
Differential Revision: D20106480
fbshipit-source-id: e33df3d0b7aa42867ee752e4c1f3a47b31ade76b
Summary:
Right now, all of our manifest parsing and evaluation is in the repo() class, but this is a design mistake. Over a repo's convert lifetime, a single repo will have many different manifests, based on branch, and location in the commit history. What's worse is that the current design makes it hard to build unit tests and new features like include evaluation.
This commit creates a whole new class called repomanifest, that represents a specific manifest (and its included files). It also has unit tests to test the various operations that the manifest performs, such as path and revision mapping. This commit does not modify the existing converter code outside of the class to use this new implementation.
Reviewed By: tchebb
Differential Revision: D19402995
fbshipit-source-id: b97dadcc595c6332f4495460618317194873a780
Summary:
In the past I saw test breakages where the stderr from the remote ssh process
becomes incomplete. It's hard to reproduce by running the tests directly.
But inserting a sleep in the background stderr thread exposes it trivially:
```
# sshpeer.py:class threadedstderr
def run(self):
# type: () -> None
while not self._stop:
buf = self._stderr.readline()
+ import time
+ time.sleep(5)
if len(buf) == 0:
break
```
Example test breakage:
```
--- a/test-commitcloud-sync.t
+++ b/test-commitcloud-sync.t.err
@@ -167,8 +167,7 @@ Make a commit in the first client, and sync it
$ hg cloud sync
commitcloud: synchronizing 'server' with 'user/test/default'
backing up stack rooted at fa5d62c46fd7
remote: pushing 1 commit:
- remote: fa5d62c46fd7 commit1
commitcloud: commits synchronized
finished in * (glob)
....
```
Upon investigation it's caused by 2 factors:
- The connection pool calls pipee.close() before pipeo.close(), to workaround
an issue that I suspect solved by D19794281.
- The new threaded stderr (pipee)'s close() method does not actually closes the
pipe immediately. Instead, it limits the text to read to one more line at
most, which causes those incomplete messages.
This diff made the following changes:
- Remove the `pipee.close` workaround in connectionpool.
- Remove `pipee.close`. Embed it in `pipee.join` to prevent misuses.
- Add detailed comments in sshpeer.py for the subtle behaviors.
Reviewed By: xavierd
Differential Revision: D19872610
fbshipit-source-id: 4b61ef8f9db81c6c347ac4a634e41dec544c05d0
Summary:
This makes `peer.close()` actually close the ssh connection if it's an
sshpeer. This affects the `clone` path to actually clean up the ssh connection
so we don't depend on (fragile) `__del__`.
I traced the code back to peerrepository.close in 2011 [1]. At that time it
seems the codebase depends on `__del__`. Nowadays the codebase calls `close()`
properly so I think it's reasonable to make the change.
[1]: https://www.mercurial-scm.org/repo/hg/rev/d747774ca9da.
Reviewed By: ikostia
Differential Revision: D19911393
fbshipit-source-id: ea640d1cd82ffcb786e22f47da8116c7f50a4690
Summary:
The added function can be used by extensions to run extra logic before the
"clone" function closes the repos or peers.
This is needed to make the next diff work. Otherwise extensions like remotenames will try to write to a closed sshpeer and cause errors.
Reviewed By: DurhamG
Differential Revision: D19911390
fbshipit-source-id: ca1364e808cebb632e051fbbdcfe4bf0dca721bc
Summary: Updates the C code to support unicode filenames and states.
Reviewed By: simpkins
Differential Revision: D19786275
fbshipit-source-id: e7aeb029b792818b1b1a9c5d3028640b56522235
Summary: There is no need to open a transaction otherwise.
Reviewed By: DurhamG
Differential Revision: D20109840
fbshipit-source-id: e47adaaeea2d7565f3629701d8de4a67d4b55182
Summary:
Verifying the changelog is quite slow and we've had more users needing
to run hg recover these days. Let's finally get rid of the verify step.
Reviewed By: simpkins
Differential Revision: D20109706
fbshipit-source-id: a512d9e11716514bce986b0e3a26347fe6afd955
Summary: Most of the fixes related to encoding in `patch.py`
Reviewed By: DurhamG
Differential Revision: D19713378
fbshipit-source-id: 66ccbd0fc7826ab2d4c05173c7e9edb96700d106
Summary: As I work, it's getting harder and harder to keep my multiple changes from introducing merge conflicts between different branches. We need to break out the repo_source's implementation in to a bunch of different files to make it easier to keep things separate.
Reviewed By: zhonglowu, tchebb
Differential Revision: D20015946
fbshipit-source-id: bf954ac581e5ca9e43c091b6b1b4c539c14471f2
Summary:
`treedirstatemap._repacked` is sometimes set in write(), but does not appear
to be used anywhere. Remove it. (I noticed this since Pyre complains about
it if you enable type checking for `write()`)
Reviewed By: xavierd
Differential Revision: D19958219
fbshipit-source-id: a55e237865160191d814ed950f69c3113bec4f64
Summary:
Add type annotations for the propertycache type.
Unfortunately at the moment Pyre still can't properly type check code that
uses this class, as it does not understand the special `__get__()` method.
It looks like support for this is hopefully coming in D19206575.
Reviewed By: xavierd
Differential Revision: D19958223
fbshipit-source-id: 0f8f15fc6935ec3feaef41d3be373a85225276fe
Summary:
Add type annotations for `dirstate.status()` and
`filesystem.pendingchanges()`
Unfortunately Pyre appears to choke when processing the `dirstate.status()`
function, and currently does not actually report type errors inside this
function at the moment. I've let the Pyre team know about this.
(If Pyre did work correctly it would report one issue since it doesn't realy
understand the `rootcache` decorator applied to `dirstate._ignore`)
Reviewed By: xavierd
Differential Revision: D19958226
fbshipit-source-id: a1cd4b9402a0a449481035cee819533c56b9b336
Summary:
This module previously used to handle deciding how a particular module should
be imported if it had multiple versions (e.g., pure Python or native).
However, as of D18819680 it was changed to always import the native C version.
Lets go ahead and remove it entirely now. Using `policy.importmod` simply
makes it harder for type checkers to figure out the actual module that will be
used.
The only functionality that `policy.importmod()` still provided was verifying
that the module contained a "version" field that looked like what was
expected. In practice these version numbers are not bumped often, so this
doesn't really seem to provide much value in checking that we imported the
correct version that we expected to be shipped with this release.
Reviewed By: xavierd
Differential Revision: D19958227
fbshipit-source-id: 05f1d027d0a41cf99c4aa93cb84a51e830305077
Summary:
Add *.pyi type stub files for most of the native C extensions.
This allows Pyre to type check functions that use these extensions.
These type annotations likely aren't complete, but contain enough information
to allow Pyre to pass cleanly on the existing type-checked locations in the
code using these modules.
Reviewed By: xavierd
Differential Revision: D19958220
fbshipit-source-id: 85dc39a16e595595a174a8e59e419c418d3531be
Summary:
This moves the build rules for the extensions in mercurial/cext into a TARGETS
file in this directory.
This will allow us to start writing `*.pyi` files that contain type
information for these modules, and store them alongside the corresponding `.c`
files. By having the build rules in the top-level `eden/scm` directory we
would have needed to keep the `.pyi` files for these modules directly in the
`eden/scm` directory instead, as the namespace for the `pyi` files is assumed
to be the basemodule plus their path relative to the TARGETS file.
Reviewed By: xavierd
Differential Revision: D19958222
fbshipit-source-id: fdc26ead16663036ffa2562a96eb1649f91cba81
Summary:
The last diff fixed this for fsmonitor. Let's skip these same paths for
non-fsmonitor.
Reviewed By: quark-zju
Differential Revision: D20014808
fbshipit-source-id: 02e3cd9aa29d9c024ba3e8e42a46e21a7c8dfc30
Summary:
Watchman may report invalid utf-8 filenames, even after they've been
deleted. Let's skip them, and print a warning.
Reviewed By: sfilipco
Differential Revision: D20012187
fbshipit-source-id: b13550918a8330ef3eb5c546105d1e054dcb7724
Summary:
Whenever remotefilelog.cacheprocess2 is set, remotefilelog.cachekey is also
set, but the later is not be present when remotefilelog.cacheprocess is. Since
remotefilelog.cacheprocess already includes the cachekey, let's not add it
twice.
This also fixes the issue where hg_memcache_client would die early due to being
passed too many arguments.
Reviewed By: DurhamG
Differential Revision: D20014792
fbshipit-source-id: 8ed6775f70cf967d1c069f8acdb5a782ee819090
Summary:
This error handling can be extremely slow: calling `self.node()` can end up
triggering a linkrev scan of the changelog, which can take over 5 minutes.
If we did want to add this back in the future we would need some sort of API
on `filectx` to try and get the node ID only if it was cheap, and that would
fail fast if this is using remotefilelog and trying to get the node ID will
require scanning the changelog.
Note that KeyError can occur fairly regularly when invoked in long-lived
commands like `hg debugedenimporthelper`. If we are asked about data in a new
commit that was added since this repository was originally opened a KeyError
will be thrown here (in which case `debugedenimporthelper` will call
`repo.invalidate()` and then retry).
Reviewed By: quark-zju
Differential Revision: D20010279
fbshipit-source-id: 0e9b4c163cb9256de57daa91eed70a3736cb1075
Summary: There are two copies of pywatchman in fbcode (!) and some changes didn't make it into the edenscm copy.
Reviewed By: quark-zju
Differential Revision: D19794480
fbshipit-source-id: bcc85e0d3efc225d94b8bfa1e433f6e9cc024643
Summary:
Some of our upcoming repo merges will make it infeasible for someone to
use a full checkout. Let's add a config that will warn users of this. It has a
few levels, starting with a suppressable hint, then a non-suppressable warning,
then a suppressable exception, then a non-suppressable exception.
Reviewed By: ikostia
Differential Revision: D19974408
fbshipit-source-id: bad35a477ad8626dbc0977465368f5d71007e2d5
Summary:
On Windows, there are *two* 8-bit encodings for each process.
* The ANSI code page is used for all `...A` system calls, and this is what
Mercurial uses internally. It can be overridden using the `--encoding`
command line option.
* The OEM code page is used when outputing to the console. Mercurial has no
concept of this, and instead renders to the console using the ANSI code page,
which results in mojibake like "Θ" instead of "é".
Add the concept of an `outputencoding`. If this differs from `encoding`, we
convert from the local encoding to the output encoding before writing to the
console.
On non-Windows platforms, this defaults to the same encoding as the local encoding,
so this is a no-op unless `--outputencoding` is manually specified.
On Windows, this defaults to the codepage given by `GetOEMCP`, causing output
to be converted to the OEM codepage before being printed.
For ordinary strings, the local encoded version is wrapped by `localstr` if the
encoding does not round-trip cleanly. This means the output encoding works
even if the character is not represented in the local encoding.
Unfortunately, the templater is not localstr-clean, which means strings can get
flattened down to the local encoding and the original code points are lost. In
this case we can only output characters which are in the intersection of the
encoding and the output encoding.
Most US English Windows systems use cp1252 for the ANSI code page and cp437 for
the OEM code page. These both contain many accented characters, so users with
accented characters in their names will now see them correctly rendered.
All of this only applies to Python 2.7. In Python 3, everything is Unicode,
the `--encoding` and `--outputencoding` options do nothing, and it just works.
Reviewed By: quark-zju, ikostia
Differential Revision: D19951381
fbshipit-source-id: d5cb8b5bfe2bc131b2e6c3b892137a48b2139ca9
Summary:
`hg rage` generates the rage in the user's encoding. Since pastes are expected
to be in UTF-8, non-UTF-8 encodings result in garbled pastes.
Similarly, the lines-dec graph renderer uses escape sequences that won't work
on web pages, and the lines graph renderer uses curved lines which don't
render very well either. Force the use of the lines-square graph renderer,
which renders well.
Reviewed By: quark-zju
Differential Revision: D19951382
fbshipit-source-id: d1a5fd2ef195658f9bf10210088031474355f168
Summary:
The Rust graph renderer expects the message to be a unicode string, so ensure
we convert it from the local encoding before passing it to Rust.
Reviewed By: quark-zju
Differential Revision: D19951383
fbshipit-source-id: 644862c63873079364cb9902bd1bb49de8aa1ab9
Summary:
As of 63c471ad8a4ba0bebd1acf70569bcdcefc3fffbf in upstream Dulwich, it
now turns commands into unicode. Unfortunately, _ssh.py in hggit sees that the
type is no longer str or bytes and thinks it's an array and puts spaces between
every letter, causing it to break.
Let's allow unicode. This broke because dulwich was recently upgraded.
Reviewed By: sfilipco
Differential Revision: D19983215
fbshipit-source-id: 059756905bf4b2c73009001b078c8723ae378246
Summary: This should get rid of the extraneous uninitialized attribute errors related to `setUp` and abstract classes.
Reviewed By: simpkins
Differential Revision: D19964487
fbshipit-source-id: 52d5a6496e372d99d4398473f9ed7672228a76f5
Summary:
Update includes to the third-party xdiff.h file to use absolute includes
from the repository root. This allows many parts of our internal build
tooling to work better, including automatic dependency processing.
Reviewed By: xavierd
Differential Revision: D19958228
fbshipit-source-id: 341dd8c94f1138cf4a387b92e1817b2a286d6aa1
Summary:
Update the C files under edenscm/mercurial/cext to use absolute includes from
the repository root. Also update a few of the libraries in edenscm/mercurial
that the cext code depends on.
This makes these files easier to build with Buck in fbsource, and reduces the
number of places where we have to use deprecated Buck functionality to help
find these headers. This also allows autodeps to work with the build targets
for these rules.
Reviewed By: xavierd
Differential Revision: D19958221
fbshipit-source-id: e6e471583a795ba5773bae5f16ed582c9c5fd57e
Summary:
Remove `thirdparty/pyre2/__init__.py` from the `libhg` sources list.
We don't compile the `thirdparty/pyre2/_re2.cc` file in the fbcode build, so
importing the `__init__.py` module from this package just triggers an
ImportError when the code tries to use it. The code then always falls back to
using the version of pyre2 included from the `fb-re2` wheel.
Dropping the `__init__.py` module from our library should simply trigger an
ImportError earlier when we can't even find this file, and the code will still
fall back to using `fb-re2`.
Including this `__init__.py` file just causes issues for type checking, since
it causes us to try and type check this file even though its dependencies are
not present.
Reviewed By: xavierd
Differential Revision: D19958224
fbshipit-source-id: 34ea8806b6ee9377f17a9318c64c91ec242225df
Summary:
Some of the methods in eden_dirstate_map.py had comments that were close to
type annotations that were added a couple years ago. Update them to proper
type comments that can be recognized by Pyre and mypy.
Also remove the unused create_clone_of_internal_map() method.
Reviewed By: chadaustin, xavierd
Differential Revision: D19958225
fbshipit-source-id: b753c030acb15cf4f8d8c536614e657ee1bcba52
Summary:
Update the `eden_dirstate_map` class to store `dirstatetuple` objects instead
of plain tuples in its `_map` member variable. Without this the `filefoldmap`
code that is used on Windows fails, as it directly accesses `self._map` and
expects it to contain `dirstatetuple` objects.
Reviewed By: DurhamG, pkaush
Differential Revision: D19841881
fbshipit-source-id: ddb7523b598cfd8ec8719a8a74446cefcb411358
Summary:
Not being able to prefetch draft parent trees should not be considered as a
fatal error.
This code path is causing trouble with narrow-heads clone:
1. Streaming clone. The client gets a changelog.
2. The client runs "pull" to get new commits. The prefetchdraftparents code path runs.
3. The client has stale remote names, and public() is lagging. `prefetchdraftparents`
will try to fetch trees at the old master, but the repo is not configured properly.
That causes a stacktrace like:
$ /usr/bin/hg --config 'extensions.fsmonitor=!' clone --shallow -U --config 'ui.ssh=ssh -oControlMaster=no' --configfile /etc/mercurial/repo-specific/www.rc ssh://hg.fb.com/repo repo
connected to hg.fb.com
streaming all changes
searching for changes
adding commits
adding manifests
adding file changes
added 1 commits with 0 changes to 0 files # <<<< No traceback if this says "0 commit".
Traceback (most recent call last):
File "edenscm/hgext/remotenames.py", line 1464, in exclonecmd
orig(ui, *args, **opts)
File "edenscm/hgext/remotefilelog/__init__.py", line 433, in cloneshallow
orig(ui, repo, *args, **opts)
File "edenscm/mercurial/commands/__init__.py", line 1615, in clone
shareopts=shareopts,
# shareopts = {'mode': 'identity'}
File "edenscm/mercurial/hg.py", line 741, in clone
exchange.pull(local, srcpeer, revs, streamclonerequested=stream)
File "edenscm/mercurial/util.py", line 621, in __exit__
self.close()
File "edenscm/mercurial/transaction.py", line 46, in _active
return func(self, *args, **kwds)
File "edenscm/mercurial/transaction.py", line 543, in close
self._postclosecallback[cat](self)
# cat = bin('6472616674706172656e74747265656665746368')
File "edenscm/hgext/treemanifest/__init__.py", line 490, in _parenttreefetch
self.prefetchtrees([c.manifestnode() for c in draftparents])
# c = <changectx b5ad643b3009>
# draftparents = [<changectx b5ad643b3009>]
File "edenscm/hgext/treemanifest/__init__.py", line 522, in prefetchtrees
self._prefetchtrees("", mfnodes, basemfnodes, [], depth)
# basemfnodes = [bin('a25f17018d7cd07f1f6bc3076f95c5980ba087a9')]
# mfnodes = [bin('ad717aac7700e783a1d84f3330d13a7731a4726a')]
File "edenscm/hgext/treemanifest/__init__.py", line 529, in _prefetchtrees
fallbackpath = getfallbackpath(self)
File "edenscm/hgext/treemanifest/__init__.py", line 2173, in getfallbackpath
if util.safehasattr(repo, "fallbackpath"):
File "edenscm/mercurial/util.py", line 190, in safehasattr
return getattr(thing, attr, _notset) is not _notset
# attr = 'fallbackpath'
File "edenscm/mercurial/util.py", line 904, in __get__
result = self.func(obj)
File "edenscm/hgext/remotefilelog/shallowrepo.py", line 42, in fallbackpath
"no remotefilelog server " "configured - is your .hg/hgrc trusted?"
Abort: no remotefilelog server configured - is your .hg/hgrc trusted?
abort: no remotefilelog server configured - is your .hg/hgrc trusted?
Fix it by making prefetchdraftparents non-fatal. This would hopefully unblock
narrow-heads rollout.
Reviewed By: xavierd
Differential Revision: D19957251
fbshipit-source-id: e65bbe6bf422776effe49055f7332ec538177a41
Summary:
This will allow us to improve our dashboards filtering out errors we are
responsible for, like missing certs on the machines.
Reviewed By: mitrandir77
Differential Revision: D19950614
fbshipit-source-id: 73503e984dfe8513a700fdcb2fc36b1618c20a4f
Summary:
Commit messages and extras can be unbounded in size. This can cause problems if users create commits with exceptionally large messages or extras. Mercurial will commit these to the changelog, increasing its size. On Mononoke, large commit messages may go over the cacheing threshold, resulting in poor performance for requests involving these commits as Mononoke will need to reload on every access.
Commit messages should not usually be that large. Mostly likely it will happen by accident, e.g. through use of `hg commit -l some-large-file`. Prevent this from happening by accident by adding configuration for soft limits when creating commits.
If a user really does need to create a commit with a very large message or extras, they can override using the config option.
Reviewed By: xavierd
Differential Revision: D19942522
fbshipit-source-id: 09b9fe1f470467237acc1b20286d2b1d2ab25613
Summary:
This parameter was originally removed in D12811551, but re-added in D12855935
due to the fact that at the time the `eden_dirstate.py` and `dirstate.py`
files were deployed in separate RPMs and could not be updated together
atomically. We now deploy these files together, so we can drop this extra
unnecessary argument.
Reviewed By: chadaustin
Differential Revision: D19913057
fbshipit-source-id: 0f0b4fde4b3124a8fc5bb568551b4e67de14d410
Summary:
Sometimes the treestate points to an unknown commit (ex. aborted transaction
might strip commits). While `debugrebuilddirstate -r HASH --hidden` is able to
fix it, it is too slow.
This diff adds treestate repair logic to the `doctor` command. It scans through
the treestate files, find a most recent `Root` entry with `p1` pointing to a
known commit.
This can be much faster than `debugrebuilddirstate` in some cases, because the
watchman clock might still be valid, and the NEED_CHECK file list might still
be small. In that case, `status` can still be fast.
Since treestate atomically updates all information needed for `status`
calculation (parents, need-check-files (or, "non-normal files"), watchman-clock
(only with fsmonitor), and stat for clean files). Reverting to a previous state
is still atomic. Correctness-wise, this is equivalent to aborting a "large"
transaction, and restoring treestate data to the state before the transaction.
It should be consistent, and the next `status` call won't mis-report files like
the dangerous `debugsetparents` command.
Reviewed By: DurhamG
Differential Revision: D19864422
fbshipit-source-id: d5d2f8b43a0c15ea2ac0e3c164edec7deeb8451f
Summary:
See the test change. Without this change repairing the changelog won't give the
user back a working repo.
Reviewed By: markbt
Differential Revision: D19864421
fbshipit-source-id: b84582c5302469828c8cfcb3db362ea82f2eea63
Summary:
Reuse utilities in the fixcorrupt extension to repair changelog.
This is better than fixcorrupt because `hg doctor` does not require a repo
object. Some messages are updated so they become more consistent with the
rest of `hg doctor`.
The main motivation is to get changelog fixed early, so other repair logic can
check if a commit hash is known by changelog or not.
Reviewed By: markbt
Differential Revision: D19864418
fbshipit-source-id: 6f95c6c6191d7db2a474a07a5278a857cf41d8e2
Summary:
Run 'edenfsctl doctor' on an edenfs repo. If there is no current repo, it might
be caused by edenfs daemon stopped running. So let's also run edenfsctl doctor
in that case.
Reviewed By: markbt
Differential Revision: D19864419
fbshipit-source-id: d2a49a126a040845b88b4883d214162326d08d8d
Summary:
We're seeing a user have issues because their username contains unicode
characters and sampling's use of json doesn't handle it well. I've not been able
to repro it unfortunately, but let's go ahead and switch sampling to use
mercurial.json.
Differential Revision: D19895419
fbshipit-source-id: a1f087d1e2c7568488c2b8d54f267bd5c8266202
Summary: There's still some issues, but it's a lot closer.
Reviewed By: quark-zju
Differential Revision: D19802023
fbshipit-source-id: da539094cbc0ba3542e4b5fd3d49f5f80455ec23
