Summary:
A recent change made delta base's be Option types, and when we
encounter a None we treat that as the end of a delta chain. In the python
translation layer code we turned this into a Key of ("", nullid) when it
should've been (filename, nullid). This broke a test.
Reviewed By: phillco, quark-zju
Differential Revision: D8768592
fbshipit-source-id: 858a8eadad8699d2e7e99040486f836710381a4f
Summary: This is useful to split dev_command_timer rows based on how many files were rebuilt.
Reviewed By: DurhamG, singhsrb
Differential Revision: D8737307
fbshipit-source-id: 18d52c0fd13184d789daaa8a74f85360554da703
Summary:
This isn't 100% finished and is missing handling some
error cases, but I wanted to put it out there to get some knee-jerk
reactions and suggestions on how to handle some things.
This diff adds a `grep.usebiggrep` config option to `hg grep`. The idea is
that we'll default this to on when the repo requires `eden` (or when we pull in
the eden specific site configuration). When run in big-grep mode, we'll first
ask big grep for the results and then compute the local differences and run
only those through the local grep process, and avoid materializing files
locally.
I'm not 100% sure if the current `repo.status` call will yield the correct
results for the case where the current rev is behind the biggrep
corpus revision.
Reviewed By: quark-zju
Differential Revision: D8416360
fbshipit-source-id: 952badb7a7ec74096b5c77cd79aa25e2327a7659
Summary:
Treedirstate could have `None` _nonnormalset or _otherparentset. Check them
before calling `.clear()`.
Reviewed By: markbt
Differential Revision: D8752216
fbshipit-source-id: 7c72e3a31a405ee2b0464ea08d93b8dd3a187597
Summary:
The recent change from delta.base being a Key to an Option<Key> needed
to update pyrevisionstore as well. I'll add this to buck build soon so we can
catch this.
Reviewed By: phillco, quark-zju
Differential Revision: D8752910
fbshipit-source-id: ecf6d2a8a74b72d10b6dc0fb24c7b9346f0f7ed8
Summary:
The gitignore implementation in core is more complete (support "!" rules)
and efficient (in native Rust). So let's drop the gitignore support in
hggit.
Reviewed By: singhsrb
Differential Revision: D8744807
fbshipit-source-id: faecace0ab0d5a0e015f50473dd1e0c232d013ff
Summary:
Currently many users use hg up to move own commits between the repos because unaware about cloud sync.
The hint will be shown on ambiguous prefix only, so it will not distract anyone.
Reviewed By: markbt
Differential Revision: D8732276
fbshipit-source-id: ae8fa10b28873134da013bc56ff9ebc33e9ed303
Summary:
Previous code format attempt (D8173629) didn't cover all files due to `**/*.py`
was not expanded recursively by bash. That makes certain changes larger than
they should be (ex. D8675439). Now use zsh's `**/*.py` to format them.
Also fix Python syntax so black can run on more files, and all lint issues.
Reviewed By: phillco
Differential Revision: D8696912
fbshipit-source-id: 95f07aa0c5eb1b63947b0f77f534957f4ab65364
Summary:
In a future diff we'll be returning data read from a pack file out as a
Delta. To avoid copies, we need to be able to return an Rc from DataPack. This
seems like it will be a common pattern, so let's go ahead and make Delta contain
its data as an Rc.
Reviewed By: quark-zju
Differential Revision: D8557949
fbshipit-source-id: 276005360bfa48e9154143dedce579a21129e976
Summary:
In a future diff we'll be serializing and deserializing metadata in
datapacks. Let's add the reader and writer functions for Metadata and some unit
tests.
Reviewed By: quark-zju
Differential Revision: D8303603
fbshipit-source-id: 7e7a7aa218c05179b205abf8b151b1488be674b3
Summary: before the waiting provide all the information to the user
Reviewed By: markbt
Differential Revision: D8734860
fbshipit-source-id: 9c0203b0879091d1d367c2ded9eccd8db4ee1bb9
Summary: simpkins found a user getting a KeyError with the former.
Reviewed By: simpkins
Differential Revision: D8730525
fbshipit-source-id: 40d80f17e0ab8b2905fc53377e501546c21feaae
Summary:
The fastpath for adjusting linknodes using the fact that a filenode ctx was
created from a descendant rev can sometimes cause lots of treefetches. Add an
option to disable it.
Reviewed By: mitrandir77
Differential Revision: D8691202
fbshipit-source-id: f196565929a2cf9b850c1c7c35f3ba6fa90f9031
Summary:
Fix an issue where pathaudit failed (see [1] for what pathaudit does) files
showing up in "hg status" with fsmonitor enabled.
This is a problem with all dirstate implementation. Treestate makes it
easier exposed, as it merges "notefiles" with "nonnormalset", instead of
replacing an exisiting "notefiles" set.
[1]: https://www.mercurial-scm.org/repo/hg/rev/2cbd27f4f3c4
Reviewed By: DurhamG
Differential Revision: D8721020
fbshipit-source-id: 7c6c8c2f202c0da4c3eeee3c9b1ce10bf7970dd8
Summary:
When watchman returns is_fresh_instance=True, it means the old clock value
cannot help watchman to do less work. Therefore the clock value needs to be
updated to make the next status call faster.
Reviewed By: phillco
Differential Revision: D8714863
fbshipit-source-id: 1eb2bf876b21bd0f17134a3b93b06b759746fd4c
Summary:
I was thinking about what we need to enable automatic restacking on non-conflicting amends (see the next diff), but I realized the --tool :abort tool I implemented in D8493065 doesn't really work well for this for three reasons:
- It can't handle the case of mergedriver having to run, which is the other reason we'd have to break out of IMM.
- It hard-codes the merge tool. The user might want to specify another tool that doesn't recreate conflicts or solves them differently.
- It'd force callers to detect if IMM will be used (to prevent an on-disk merge hapening by mistake). A flag can implement this much more easily.
- As I learned when writing D8493065, it would require hardcoded logic in every command anyway to support the non-IMM case. (Just raising an abort will leave most commands with an interrupted state.)
So, I think we should replace it with this flag to make auto-restacking work very reliably (we could add it to graft, too). If there was some big demand for --tool :abort we could always add it back in the future.
Reviewed By: quark-zju
Differential Revision: D8701897
fbshipit-source-id: ea3b92d0a224a8ce43edb120b53bec241d92a61d
Summary: This will let us print more helpful/precise error messages.
Reviewed By: singhsrb
Differential Revision: D8699551
fbshipit-source-id: 060058eff925d94ffa8e6b4432c4e51bd93b2945
Summary: This config was needed when IMM restarts were long and painful, but now that they're not, this clunky and inaccurate hueristic can be removed, which simplifies the control flow a bit.
Reviewed By: singhsrb
Differential Revision: D8700868
fbshipit-source-id: 0aeac26e5c41f7f0260ee826f05a8f9047dbad5a
Summary: This configuration is set everywhere, and the spectre of stateful preprocess() functions was a Facebook-only abomination that hopefully won't reoccur. Delete this config.
Reviewed By: quark-zju
Differential Revision: D8700125
fbshipit-source-id: b5ef163f988b7bb79310275d54afea3eab2c23a2
Summary: The new logic is on at 100% without problems, so axe the old.
Reviewed By: singhsrb
Differential Revision: D8698417
fbshipit-source-id: 0126c61c7bbfa49609354415f7cc61e861c1f2d5
Summary:
The manifestlog has some mutable packs that it persists and manages the
lifetime of. As long as every tree writer uses these, we'll only get one new
pack per process/transaction.
The prefetch code was not using these though, so things like 'hg log -p' would
produce many packs over the course of many fetches. This change fixes it to
always write to the same pack.
Reviewed By: phillco
Differential Revision: D8670684
fbshipit-source-id: e56590c0ced84a99a06a94028d6a4e14bcc22ab6
Summary:
We already had mutable local packs, which were used to allow all local
writes to go to the same pack. Let's add an equivalent mutable shared pack. A
future diff will change shared writes to go to these packs.
Reviewed By: phillco
Differential Revision: D8670682
fbshipit-source-id: cbe3ab40b30f5ff53ece23914a95ffeba8b03ec3
Summary:
ManifestLog._mutabledata/historypack is used to persist a single set of
mutable packs for all local writes in the process. In a future diff we'll be
addeding the same concept for shared packs as well, so let's refactor the
current logic to explicitly be named _mutablelocalpacks.
Reviewed By: quark-zju
Differential Revision: D8670679
fbshipit-source-id: 8b1a04490c71525eca9466a7288dd7fd23991089
Summary:
D8464848 added code to invalidate merge drivers after running them, for reasons described in that diff.
But it turns out deleting from sys.modules is not enough, because Python will occasionally write a `.pyc` of the original hook, and re-use it when loading again, thus bypassing the invalidation.
This doesn't happen at all on my Mac, but happens approximately every other time when running on CentOS. Thanks to DurhamG who discovered the flakiness.
This fixes the flakiness of the test, and AFAIK this would be a problem in the wild, too, though we haven't heard of it yet.
Reviewed By: singhsrb
Differential Revision: D8675720
fbshipit-source-id: bec8fec6af9a362db1f0dd5d262932cbda6137ff
Summary: Break the cycle by using local variable.
Reviewed By: chadaustin
Differential Revision: D8686397
fbshipit-source-id: e1846d33472066466814ab2c0c666f72b8393ff3
Summary:
A reference of `repo` is kept in the `finalize` closure. That forms a cycle:
- finalize needs repo
- repo.transaction needs finalize
Break the cycle by making `finalize` use `self` passed to `repo.transaction`
instead.
Reviewed By: chadaustin
Differential Revision: D8676286
fbshipit-source-id: e27bed996883e2bf22cdeec41fcf1ab3e771b208
Summary: When wrapping the repo object, use local variable if possible.
Reviewed By: chadaustin
Differential Revision: D8676284
fbshipit-source-id: d35d9b7b9357a8e16d4eb10032911ecbdc34b907
Summary: When wrapping the repo object, use local variable if possible.
Reviewed By: chadaustin
Differential Revision: D8676283
fbshipit-source-id: 627e5fa4cc126efff42f29cf463fa90b97d9a0e0
Summary:
fastmanifest uses a `uiproxy` semi-global variable to workaround a chg issue
where `uisetup` received an out-of-date `ui` object.
That's no longer necessary D7840237. Removing the `uiproxy` global variable
fixes the memory leak.
Reviewed By: chadaustin
Differential Revision: D8676285
fbshipit-source-id: a5e042e4f61460ce53d43a231f07d70f27a52f19
Summary:
The blackbox extension uses a global variable for `lastui` in case certain ui
objects do not have an associated repo. That `lastui` reference makes the `ui`
(and `repo`) harder to be dropped. It also sets `ui.repo` which could form a
cycle if `repo.ui` is that `ui`. Fix it by using weakref.
Reviewed By: chadaustin
Differential Revision: D8675908
fbshipit-source-id: 7d0c083cc5c7a3ac134379732de1e0c408a1d5fc
Summary:
Continue with the sampling fix. hgsubversion is the second trouble marker.
It uses similar patterns (reference to non-local variables in closure).
Fix it by using local variables.
Reviewed By: chadaustin
Differential Revision: D8675439
fbshipit-source-id: bff7a827608fbf4a2662e6e9956ddb733c6a3bb5
Summary:
When debugging T30931730, I created a samll repo with almost all extensions
disabled. Then there is no leak report for the eden importer. Then I start
bisecting the extensions, and found the sampling extension is one of the
troublemakers. This is similar to a fix in hgsubverison [1].
This does not reproduce outside the eden importer because the atexit handlers
are consumed by `dispatch.py`. Regarding on whether eden importer should also
process atexit handlers or not remains a question. Anyway, avoiding cyclic
references shouldn't make things worse.
[1]: 3c4fbf59e5
Reviewed By: chadaustin
Differential Revision: D8675401
fbshipit-source-id: cf87869d2006d5166c697c2a86b65ece6cbe8c06
Summary:
The windows compiler doesn't support this, so let's just hard code our
default value.
Reviewed By: singhsrb
Differential Revision: D8679518
fbshipit-source-id: 031a6e06d12c4980756e60cdeb76b6828ed8f775
Summary:
With the `globalrevs` extension enabled, Mercurial starts adding a
strictly increasing revision number to each commit which is accessible through
the `globalrev` template.
Reviewed By: DurhamG
Differential Revision: D8364387
fbshipit-source-id: 84bd43f56b9cb1c775cffb875501dd64d10ffa42
Summary:
Adjust link node needs to read the node for a given file in a manifest,
and it can inspect manifests far back in time. The tree fetching code attempts
to download the entire tree, which makes each lookup take 40+ seconds in some
situations. Let's tweak adjust linknodes so downloads during it will download
trees one-by-one as needed.
Let's also record how long adjust link node takes in the scuba table.
Reviewed By: quark-zju
Differential Revision: D8629725
fbshipit-source-id: 4f1f9a3694c45a10a53f23ee081ba5f3c9d296b6
Summary:
We're experiencing cases where users are pulling down far more trees
than they need. Now that we can download just portions of a tree, let's add a
depth limit to the fetch code path. This will let us avoid downloading trees
that are never used.
The default is still to fetch all trees, but future diffs will make certain code
paths download only portions of trees.
Reviewed By: quark-zju
Differential Revision: D8523217
fbshipit-source-id: 066c5aef14b00ea0a366cf43e8a89ee053f82d22
Summary:
We're experiencing cases where users are pulling down far more trees
than they need. Let's enable the ability to download a certain depth of trees.
In this diff we just implement the actual iteration limiting in native code. In
a future diff we'll make the appropriate python changes and add a test.
Reviewed By: quark-zju
Differential Revision: D8523215
fbshipit-source-id: ccfeadee1525b86b1a5578631dbafe666de41f4c
Summary:
Previously, we could only ask `logginghelper` to log some `section.name`
config option via `ui.log` under the `name` name. This might be not enough
when section keys are generic and only make sense when context is available.
The entire list of config options to log was also supposed to be provided
under a single config option, `logging.configoptions`, therefore it was
impossible to extend it in the included config files, just completely
overwrite.
Let's solve this by dedicating the entire `[logging]` config section to
the purpose of defining other config options to log, where each key would
be a name to use and each value would be a `section.key` of the desired option
to log.
Reviewed By: DurhamG
Differential Revision: D8639436
fbshipit-source-id: 442792c2ea11cb4c9e7191b92cd4a8c0c67e9f27
Summary:
Remove the code from the highlights extension that tries to eagerly load the
pygments lexers. This prevents commands that load the highlight extension
from failing in some circumstances.
This code was added in https://phab.mercurial-scm.org/D1619 since it avoided
"really weird traceback problems caused by demandimport messing with some of
the lexer plugins." It also claimed that eagerly loading these lexers
shouldn't affect most users since the highlight extension is only relevant for
servers. However the highlight module is loaded by other client-side commands
in several cases (for instance, "hg help"). When combined with the strict
foreign module import checking this causes this extension to get loaded
triggering this code.
This `find_plugin_lexers()` is using pkg_resources internally to find its
dependencies. This fails if some of the dependencies were installed as plain
python packages rather than as setuptools packages or eggs.
Reviewed By: quark-zju
Differential Revision: D8656400
fbshipit-source-id: 0b16852208e56823c5f8ab055b69020ac338447b
Summary:
When we apply draft commits to the repository, through
pull/unbundler/etc, we want to prefetch their public parent trees. If we don't,
when the draft commit is accessed it has the root node, but not some of the
children, and therefore attempts to download all the necessary trees ondemand
without any base trees. This is very expensive.
Let's prefetch the public parents, which should be relatively cheap since it can
delta against a nearby tree, and which is probably required anyway since
applying draft commits usually indicates they will be accessed soon.
Reviewed By: quark-zju
Differential Revision: D8654617
fbshipit-source-id: 63ee1e958fc0a0abdb8fa7bdb8c5d9e01181b6a7
Summary:
A future diff will want to use the base wirepack.receivepack with some
existing data and history packs, so let's refactor it so it can be used this
way.
Reviewed By: phillco
Differential Revision: D8404524
fbshipit-source-id: b1ab053870042933a2e2125f9bd58175479da255
Summary:
This commit just moves out the updates to the revision_references
table to a separate method so that it can be easily wrapped around.
Reviewed By: DurhamG, phillco
Differential Revision: D8406981
fbshipit-source-id: 8046a3e5b41dcf6c2f569699ac03de78611ba0bd
Summary:
p4fastimport has been replaced by p4seqimport months ago, on top of that previous assumptions have changed that might make p4fastimport not work (e.g. allowing import to connect to LFS).
This change cleans up a lot of unused code and a bunch of tests that no longer make sense.
Reviewed By: zhh95
Differential Revision: D8525286
fbshipit-source-id: 91d33e7530bf6df1e1ec92fae3acde0345230247
Summary:
this command will be use in fbclone
it does sync and join if a person has been used already the commit cloud workspace in that repo
Reviewed By: markbt
Differential Revision: D8640667
fbshipit-source-id: 9c8d96497cf7930b995a49c8bdaebe7048e0c1e8
Summary: show commit cloud workspace state and infinitepush state as pretty jsons
Reviewed By: farnz
Differential Revision: D8611360
fbshipit-source-id: 97c0f38cd25bc0cc73f9a41d474b94234e463d3c
Summary:
Use the registrar to register template keywords and revset predicates, and
remove code for running the extension in old versions of Mercurial.
Reviewed By: farnz
Differential Revision: D8633454
fbshipit-source-id: aa0ebf86ff644072a3e78df8142dd8da7a188309
Summary:
Assume the stack:
```
o - C
|
o - B
|
o - A
|
o - Root
```
then if we need to metaedit commits A and B the stack becomes:
```
o - C' (metaedited)
| o - B'
| |
| o - A' (metaedited)
| /
o - Root
```
This bug affects also Jellyfish while submitting the stack of commits where some commit in the middle is already assotiated with Phab diff.
The bug is caused by the logic where we firstly metaedit and rebase the commits to metaedit in their topological order and only after all their descendants. So for now the given commits are processed with their descendants in topological order to track new parents for each commit.
Reviewed By: mitrandir77
Differential Revision: D8591897
fbshipit-source-id: 330a598e084af60cff42255510aca420bfe389de
Summary:
Add new revsets for finding remote bookmarks and remote branches by their exact
name, or by regular expression.
Reviewed By: quark-zju
Differential Revision: D8588923
fbshipit-source-id: 11e6d3636a0cccc5145c7dbd8824135b51dcbab3
Summary:
Add a config option `fsmonitor.detectrace` and environment variable
`HGDETECTRACE`. Once set, perform an additional query to check if there are
writes to the working copy while `wctx._buildstatus()` is being calculated.
Also enable the race detector for all fsmonitor tests.
Reviewed By: wez
Differential Revision: D8597957
fbshipit-source-id: 5d83d529641325dc8d5e72ad059eed8db27d1f2a
Summary:
For some reason, some users still have `perftweaks.cachenoderevs` enabled.
That actually slows down things because `clindex` is expected to make node
lookup faster without special hacks. So let's disable
`perftweaks.cachenoderevs` if clindex is enabled in config. This would also
allow us to clean up related configs in Chef (blocking D7826336).
Reviewed By: DurhamG, singhsrb
Differential Revision: D8626199
fbshipit-source-id: 69a6f22ee25ff0a00e870f7d07c73ca0431720c9
Summary:
use -w option instead of -p for keychain command security add-generic-password
-p Specify password to be added (legacy option, equivalent to -w)
-w Specify password to be added
options are the same, but -p seems to be legacy one
Reviewed By: DurhamG
Differential Revision: D8481013
fbshipit-source-id: 7c2c47f0b4e4737612416449c2a7c000ca09ab29
Summary:
Let's advertise sparse to all the people who still need to hear about this.
This is behind config know. I think it's reasonable to enable it for ovrsource
and fbsource (possibly with different tresholds).
Reviewed By: quark-zju
Differential Revision: D8591973
fbshipit-source-id: 99d22705eb81e0b646b0c1537ddec395322069e6
Summary:
provide sockpath in the config, rather than calling `watchman get-sockname` command from python.
the config variable was already deployed on all mac
this improves hg st by 250-300 ms
(the logic gives more priority when env var if specified (the env var override is usually used for testing))
the logic has a retry mechanism if we can't connect to the socket path specified in the config
next step: deploy config on dev servers (but we will have only few ms speed up there)
we use %i (i for identity) notation for user
[298] → hg conf | grep sockpath
fsmonitor.sockpath=/opt/facebook/watchman/var/run/watchman/%i-state/sock
Reviewed By: farnz
Differential Revision: D8549130
fbshipit-source-id: 31df9ebf3eb5133d3e6f64794022cda1b0fadec4
Summary:
This config has been True everwhere for a while, and seems uncontroversial. Let's fold it in everywhere as True.
This simplifies a test case.
Reviewed By: DurhamG
Differential Revision: D8578289
fbshipit-source-id: f66a6427e435d80750b4da14358c5e4f0528b6f1
Summary:
D8560049 made `{reviewers}` work with commit template, but it breaks empty
`committemplate.reviewers` with `abort: "" not in template map`. This diff
fixes it.
Reviewed By: DurhamG
Differential Revision: D8596389
fbshipit-source-id: e1c046d59097181ef5c8d56a1cffc7c2407f0ba5
Summary:
Treestate tracks untracked files while treedirstate doesn't. The
treedirstate importmap logic changed by D7912557 will remove untracked files
from the tree. But it missed other places like nonnormalset, otherparentset
and copymap. This patch changes it to make sure paths in those sets are
tracked.
Reviewed By: DurhamG
Differential Revision: D8560022
fbshipit-source-id: 8ac39bfe634804ea94125fe42e11382acae35d94
Summary:
Previously `state._usetreestate` is a value set in `state.__init__`. In the
treestate migration case, `state._usetreestate` needs to change accordingly.
Therefore let's use a property to always calculate that value from the
source of truth.
Reviewed By: DurhamG
Differential Revision: D8560019
fbshipit-source-id: a313f603275321b959502e7f2c528c4b54a11ce9
Summary:
Revset predicates that work substantially through `subset.filter` should have a
higher weight than the default value of 1, to prevent the optimizer incorrectly
optimizing operations like:
```
reverse(ancestors(node, limit)) & filterpredicate()
```
in a way that filters all the revisions in the repo according the predicate
before performing the dag search.
Reviewed By: ikostia
Differential Revision: D8588027
fbshipit-source-id: 57be642761af013be8c8aa27ec126b3919c34256
Summary:
The expression `::revs & age("<x")` can be slow, as all ancestors must be
analyzed in case there are commits with ancestors that have later dates. Most
of this time this is unnecessary, as commits are usually monotonic in their
dates.
The `ancestorsaged` revsetpredicate evaluates to all ancestors within an age
range, optimized as if commit dates are monotonic.
Reviewed By: ikostia
Differential Revision: D8489319
fbshipit-source-id: 9b9055019c2afa70497a9df02c6fe3504e2c381a
Summary:
In the recent refactor we changed the definition of "is this tree
local" on the server to also include if the tree didn't exist in the store,
since then it might be an ondemand conversion from a flat manifest in a bundle.
This caused issues on peer-to-peer pulls, since most trees don't exist locally.
Let's fix it by only considering a missing tree to be a local tree if we're on
the server.
Reviewed By: quark-zju
Differential Revision: D8586779
fbshipit-source-id: ea8c8ee3e2a4dc054fae9caa122cfc733df1ab81
Summary:
The treemanifest manifest revlog store was keeping all the revlogs it
had ever opened. This was causing unnnecessary memory usage. Let's use an lru
dict instead.
Reviewed By: singhsrb
Differential Revision: D8579992
fbshipit-source-id: a0df30401714c93eebe0448b0ce92571f704bd96
Summary:
It seems the datapackstore c++ implementation is very leaky, which is
causing memory issues on the server. We're in the process of rewriting the
datapack logic to be pure rust, so let's just disable this feature for now.
Disabling this slows down tree downloads by 25%.
Reviewed By: quark-zju
Differential Revision: D8580519
fbshipit-source-id: 7e42e7ff211008d5461f95386da019ba5696e07b
Summary:
The template keyword only works for committed changesets. That conflicts
with committemplate. So special case the working copy case and make it work
if committemplate.reviewers is set.
Reviewed By: liubov-dmitrieva
Differential Revision: D8560049
fbshipit-source-id: 1427d0b72e023741be7012dd6eb4d51b0b595f39
Summary:
The `repos` file contains paths to repos so `hg gc` can scan them. The
current implementation basically writes a line per `hg pull`, and `repos`
can grow unbounded. Let's do a check before appending to that file.
Reviewed By: ryanmce
Differential Revision: D8416210
fbshipit-source-id: 0624bfbd6eb7d71dd749626e96063cb10f55bb7f
Summary: Commit cloud state is stored in the store, so the wlock is not necessary.
Reviewed By: liubov-dmitrieva
Differential Revision: D8121600
fbshipit-source-id: f96f4764e3cf0637cd45a0bc83d1227318ac9083
Summary:
`/` in this case is semantically not a dir seaparator, but a bookmark namespace
separator, so let's use `posixpath`, rather than `os.path` to combine parts.
Reviewed By: StanislavGlebik
Differential Revision: D8477029
fbshipit-source-id: 964b7d3f8a560fb73dfaaa39757c091908af77c5
Summary:
Flat manifest backfilling (which is used when reverting from treeonly
mode) was failing when the client had public heads that didn't exist on the
server (for instance, if they pulled from svn directly or if they had old
commits that had been stripped from the server). This was breaking the pull.
Let's fix it so we never send unknown heads to the server during the backfill
pull.
Reviewed By: phillco
Differential Revision: D8526027
fbshipit-source-id: 8d5bf67d2022dbc24facd0b96e001c8c81a0a2da
Summary: This adds a merge tool to only rebase if there aren't conflicts, which I think will be widely useful. It's for IMM only at the moment.
Reviewed By: DurhamG, quark-zju
Differential Revision: D8493065
fbshipit-source-id: 32ccc0e53af27d275a56bcac80861a421d7abfc6
Summary:
This was added to ease the IMM rollout, but today IMM rarely raises new exception, and this code fires on real errors that should be raised -- you can see here this lets us remove many hacky workarounds.
Some of the exceptions that didn't have carve-outs were:
- interrupted update
- source and destination form a cycle
- rebase in progress
Reviewed By: DurhamG, quark-zju
Differential Revision: D8522466
fbshipit-source-id: b407807ea059a0861fb3caca1db8699242342075
Summary:
This is a quick bug fix for the situation xixixao discovered:
You rebase commit B onto C while having A checked out, and the rebase creates merge conflicts.
In that case, the command aborts complaining of the uncommitted changes. But, the rebasestate isn't properly removed. The user might logically run `hg rebase --abort` to clear it, but that does an
`update` with `force=True` which can cause work to be lost.
This is just a quick bug fix. The right solution is to make an inmemory `mergestate` and not write it at all if rebasing in-memory.
Reviewed By: DurhamG
Differential Revision: D8517289
fbshipit-source-id: a62c99ccb2de1be102f9c23cd737794184c4fde8
Summary:
When building a bundle that will be used for a pull that can't use the linkrev
fastpath, we need to process the manifests of the commits for which we will
send files so that we can correctly set up their linkrevs. Currently this is
sometimes happening for all commits, which can be very slow when pulling a
large number of commits.
First, make it clearer what `LocalFiles` and `LocalTrees` means, and make it
easier to test for trees being local.
Second, when processing manifests looking for files that will be included in
the bundle, don't process non-local trees if we are only interested in local
trees.
Third, correctly determine whether to send all, local, or no trees or files
depending on what combinations of treemanifest and remotefilelog the client and
server support, and what operation is taking place.
Finally, we aren't passing `b2caps` through to `changegroup.makestream` in the
changegroup part generator for bundle2 parts. This means the above test for
whether the client supports treemanifest always failed.
With these combined changes, a pull of a specific public revision (so not
following the fastpath) where the client supports both remotefilelog and
treemanifest, will result in the server using `LocalFiles` and `LocalTrees`,
and therefore not processing the manifests of public commits looking for which
files to send.
Reviewed By: DurhamG, ikostia
Differential Revision: D8446137
fbshipit-source-id: 9a4247af75fe79c1962a29fcb8bf0181868699f4
Summary:
recent refactoring broke user experience if token is not found
not found is actually an error, rather than an empty result for security command
Reviewed By: markbt
Differential Revision: D8489287
fbshipit-source-id: 5ee6e821ce79877b43f93cb86b6935294b9e4a67
Summary: Move the log statements up so it becomes effective for treestate.
Reviewed By: markbt
Differential Revision: D8458183
fbshipit-source-id: af943c46dc3a3b0bd403476112473095d33b3650
Summary:
Let's make it clear for the user what background process is running and how long
I think it is really nice because the user can see if it is stuck and also have direct opportunity to kill it
Reviewed By: markbt
Differential Revision: D8476817
fbshipit-source-id: fefaef91e62b5e6199801a9e010a360e26ee6824
Summary:
Implement gc for treestate. Dirstate writes are protected by wlock. So it's
fine to do a gc inside the write method.
Reviewed By: markbt
Differential Revision: D8394736
fbshipit-source-id: 672087a2c14b855d1ca0449eb5b7d92d65bba418
Summary: Similar to treedirsate, if the file size exceeds configured limit, do a repack.
Reviewed By: markbt
Differential Revision: D8394735
fbshipit-source-id: 21a73cd32b33ed462bab8050c3d42a5e7545d3a4
Summary:
cloud sync:
when it is executed manually it doesn't have any sense to let user wait 30 seconds and look at nothing (there were no any message before 30 sec)
in the world of scm daemon, it is a normal case that cloud sync is already running,
let just tell it to the user and exit
cloud sync:
for background / scm daemon runs timeout is still the same
Reviewed By: markbt
Differential Revision: D8449219
fbshipit-source-id: 2b5c0c6874e23bcd71ca74b2a5231df3ae581e3b
Summary:
inform user that backupdisable command doesn't affect the current running
process if any exists
Reviewed By: markbt
Differential Revision: D8471761
fbshipit-source-id: 338b6d9e0332f5c845f26cbfddbf1d1c89f0ac41
Summary:
cloud sync can trigger another cloud syncs from itself when pull commits
disable it, this is the same setting we use for pullbackup
Reviewed By: markbt
Differential Revision: D8477217
fbshipit-source-id: 656fff11870d05c33aabf284fc9b4070bd5a29ea
Summary:
This fixes a subtle bug related to mergedrivers and IMM.
If your DAG looks like this:
```
dest (new mergedriver code)
|
| A (current commit)
|/
base (old mergedriver code)
```
And if you use IMM to rebase `A` onto `dest`, the wrong thing can happen because parts of `base`'s mergedriver are still loaded.
This only affected certain types of driver changes -- adding or removing parameters between files in the driver, or a driver change that also mutated the working copy (e.g. renaming a file checked by `preprocess()`). And, it only occurred if you had `A` (or some other change between `base` and `dest) checked out.
So, evict the old driver files, and all is well.
The right long-term fix is to probably implement our own Python loader for the mergedriver that reads all driver files from either the `source` or `dest` changectx not `.`. But, that's a bigger change.
Reviewed By: quark-zju
Differential Revision: D8464848
fbshipit-source-id: 8f03a4c99418aa26c3890989d19dd908b026c328
Summary:
implement in the easiest way, we will implement in a better way later
this will reduce confision when we will launch scm daemon
Reviewed By: farnz
Differential Revision: D8450341
fbshipit-source-id: 1c6bedbcf68224dd1e61ef4ec5cd04f26e76453a
Summary:
Regression caused by D7909172. This could trigger slow "status" calls 100%
for treestate, and cause race condition for non-treestate cases.
Reviewed By: phillco, singhsrb
Differential Revision: D8458182
fbshipit-source-id: 6a2e9939d83f55be4b0597b6e15932c9219aac4b
Summary:
This commit simply adds a method for checking if the hgsql extension
is bypassed. This will be used in a later commit for validating the repository
configuration.
Reviewed By: DurhamG
Differential Revision: D8364386
fbshipit-source-id: 70cc08c55339dc4a47ae17b30c131bbd2167e144
Summary:
This commit moves the logic concerning creation of commit during the
graft step in pushrebase to a separate method. This allows for easily wrapping
around the commit to add custom logic. In particular, this can be used to add
extras for the commits that will be created using the pushrebase.
Reviewed By: DurhamG
Differential Revision: D8364383
fbshipit-source-id: 60f845ec3f4f0b4d3799f4385ce49c6fd78cdda2
Summary:
This commit just introduces a method to check whether non pushrebase
commits are blocked. This will be used in later commits.
Reviewed By: DurhamG
Differential Revision: D8364384
fbshipit-source-id: ca0fb39e7925ce8dc7374bf9eb26f9c47506a67c
Summary:
The new treestate was designed to store fsmonitor state. Use it to make
fsmonitor and dirstate state consistent, and avoid fsmonitor state
invalidation.
The "fsmonitor identity" check was removed as we now rely on the dirstate
identity check - dirstate and fsmonitor state must be updated consistently -
both updated or neither updated.
Since this is the first dirstate that tracks "untracked" files, several places
are adjusted (ex. dmap.dropfile, dmap.keys) to take the new untracked files
into consideration.
Reviewed By: wez
Differential Revision: D7909172
fbshipit-source-id: 05fd64b25c67ae4b07bc8cfee2731c748205975e
Summary:
The previous logic restarted the *entire* rebase if one commit had merge conflicts or needed a mergedriver run. This was unnecessarily slow, plus the abort triggered a full repo scan.
Also, this logic is actually simpler and easier to reason about when done here, vs. `rebase()`. We can delete all the wrapping that function does after this rolls out.
This is config-gated, so we can roll it out gradually. Old config = old behavior.
Reviewed By: quark-zju
Differential Revision: D8272566
fbshipit-source-id: 9e598eea10c8a634497cd211a468dd2f58c5ff47
Summary: This just moves the logic around rebasing a single commit into a function, _rebaseone, so we can call it twice in the next diff.
Reviewed By: quark-zju
Differential Revision: D8272564
fbshipit-source-id: 38a15c099bb75d3a6b015fd22453c3732a6b21f5
Summary:
security command normally show a GUI prompt to allow you to type your password if the keychain is locked
if it is unable to do so, it returns 36 and empty stderr
it is ok that it is failed, because it happens due to security reasons,
the problem is only that stderr is empty, so the reason of failure is not clear to user.
I didn't find any option in the man of security command itself. So, I checked the status code.
Reviewed By: markbt
Differential Revision: D8418363
fbshipit-source-id: 0c8f0bf55f588822e50395c3ec8bfa8d3188ecf7
Summary:
D8410266 introduces the use of `chrono_literals` namespace, which requires
C++14. Since we don't use C++14 otherwise, let's not use it here as well.
Reviewed By: farnz
Differential Revision: D8418398
fbshipit-source-id: c2243c2cdf51587cda698bf7c2e96691cea2d305
Summary:
Avoid transient partially synced stated by updating the bookmarks and
obsmarkers as part of the pull command, if there is one. Additionally,
prevent remotenames bookmarks from being updated in cloud-sync pulls that
happen to include public commits. That's preferable, as usually this will only
be a subset of remote bookmarks, and this can give confusing results.
Reviewed By: DurhamG
Differential Revision: D8401117
fbshipit-source-id: 5ee646d8c7e49ea70d3ec5fd7df5c9009de7bcdd
Summary:
We get the list of obsmarkers to sync after we have pushed the commits. In
that time, new obsmarkers could have shown up, which shouldn't be synced until
the next sync operation.
Improve this by getting the obsmarkers just after we get the heads and
bookmarks. This still has a race, but the window is now much smaller.
It also means we can ensure we sync even when all we have is some extra
obsmarkers.
Reviewed By: DurhamG
Differential Revision: D8397739
fbshipit-source-id: e29890aa250d58bfc3c3c2a056a01d287c053605
Summary:
This fixes how DatapackStore::refresh() computes if it needs to perform a
refresh.
Previously this code was using `clock()`, which returns the amount of CPU time
used by the program. This seems like an undesirable metric to use: the amount
of CPU used by the current program doesn't seem like it should be relevant to
whether the files on disk need to be re-examined or not. If the program has
slept for an hour we probably still want to refresh the files even though no
CPU time has been consumed.
As a result of how the computation was performed, markForRefresh() also would
not trigger a refresh if the program has used less than 100ms of CPU time so
far.
This was breaking some of the eden tests which create a repository pull some
packs into it, and then immediately refresh the store. If all of this happens
in less than 100ms the DatapackStore code would not actually refresh the
store.
Reviewed By: chadaustin
Differential Revision: D8410266
fbshipit-source-id: f08d317b261ccfe89cff45bf3e511a2b87cf9fb5
Summary: This allows us to dogfood the new treestate.
Reviewed By: markbt
Differential Revision: D7912559
fbshipit-source-id: bfb30cd9989460b0eae859aeed402bd92776ab0a
Summary: This makes it easier to support format version 2 - treestate.
Reviewed By: markbt
Differential Revision: D8394003
fbshipit-source-id: ef341fc6d5dd6ba34ad57a98dc3ed63e03d5c1c4
Summary:
Previously, there are 2 configs: `treedirstate.useinnewrepos` and
`format.usetreestate`. They are both related to dirstate format and conflict
with each other. This patch unifies them into a single config
`format.dirstate`.
As we're here, merge `test-fb-hgext-treedirstate-x.t` to `test-dirstate-x.t`
if they were previously copied from `test-dirstate-x.t`
Reviewed By: markbt
Differential Revision: D8393878
fbshipit-source-id: 57abeea22ce732d93205e4d4308923afa90693f4
Summary:
When requesting trees from the server we try to provide a base tree
that we already have. In the hybrid manifest days, when we were fetching
manifest X, we would look up the linknode for X in the flat manifest revlog,
then scan the changelog up and down from that linknode. In the treeonly world we
don't have that revlog anymore though, so the current algorithm just searches from
the tip. This ended up being a really bad algorithm, since if you have tip and you
request tip~1000, then tip~2000, then tip~3000, you end up basing them all
against tip and redownloading a lot of data. This makes hg blame and hg log -p
super slow.
Let's change the algorithm to base off of a hint linknode, and let's change
changectx to set that hint every time it attempts to read a manifest. In manual
testing this significantly sped up hg log -p
Reviewed By: phillco
Differential Revision: D8399613
fbshipit-source-id: 771a94ee8b82be682ea0091b8d6c0fcd5f4e6646
Summary: as titled, so that we can `hg log -T {reviewers} --rev .`
Reviewed By: quark-zju
Differential Revision: D8389944
fbshipit-source-id: b3b194b7027bc831d5eca42dfcb4300745d6eb12
Summary: sunshowers hit an exception (either an upgrade race or a module deinitialization) and while it was spurious, it can't hurt to run this logic in a try/catch anyway.
Reviewed By: ikostia
Differential Revision: D8360101
fbshipit-source-id: e581bdeb69e22571f547633b7219d6886cab1aa7
Summary:
If the repo has treemanifests, we can use this to do the case check faster.
Note that with treemanifests we only find the first path component that
conflicts (subsequent conflicts won't be found), as these conflicts occur in
different trees.
Reviewed By: farnz
Differential Revision: D8332426
fbshipit-source-id: 243dab9bf711c6fdb62a0e0ba4346faea3b6b0ad
Summary:
Add a new function to `cstore.treemanifest` to list the contents of a directory
in the tree manifest.
Reviewed By: ryanmce
Differential Revision: D8332425
fbshipit-source-id: db74f11a691cd6b8ac7d51aec6b7da8149c22027
Summary: This will allow us to have json output
Reviewed By: quark-zju
Differential Revision: D8332945
fbshipit-source-id: 3ad39c4df4fe0fbe038d277eabeb18fed10a4bb1
Summary:
Detect the case when we're writing an commit that already exists and skip
writing the history entries.
Reviewed By: DurhamG
Differential Revision: D8363935
fbshipit-source-id: d8db6233eb0261b49e3903c0345c898f2abffa12
Summary:
It's legit for a file name to contain `%s` or `%r` strings. Previously,
blackbox always expect the first argument to be the "format string". That's
inconvenient and it's easy to just pass a string containing the troublesome
`%s` to `ui.log`.
Let's just do not even try formatting strings if there is only one argument,
and if there are multiple arguments, fallback to concatenate them if they
cannot be formatted.
Reviewed By: DurhamG
Differential Revision: D8364130
fbshipit-source-id: 75b2d0e0a460b9a86d4ecd6ecfbb77c0c0fbe98c
Summary:
Add a config option `format.usetreestate`, which would turn on the new
treestate for new repos.
Treestate does not need manually maintaining "copymap", "otherparentset",
"nonnormalset". The one single tree is the source of truth of everything.
So some code like `copymap.pop` etc. are disabled on treestate path.
Reviewed By: markbt
Differential Revision: D7909175
fbshipit-source-id: 96a405fccd4099cefc152ec17204cb5407efbe7e
Summary: return code check was not implemented before but it is important
Reviewed By: markbt
Differential Revision: D8348689
fbshipit-source-id: 69948091832d4c954284e0c999da05c4980d8228
Summary:
Let's roll this out gradually with opsfiles instead.
Also fix an inverted logic test.
Reviewed By: singhsrb
Differential Revision: D8339739
fbshipit-source-id: 2700d4a16ee38280d127e2a4ee7458bdff32f38c
Summary:
Some folks might attempt to load this extension despite it
not being useful with eden. The extension knows not to hook in to
the repo when eden is in used, but the various commands to manipulate
the sparse config do not.
This diff adds a little helper function to show a more reasonable
error message than an ugly stack trace when attempting to use them.
Reviewed By: simpkins
Differential Revision: D8328229
fbshipit-source-id: d7cc4c7047b11f8da96e26ab70bd522b52546151
Summary:
This fixes the issue where every merge conflict situation immediately took 30s - 1m due to the "scanning working copy" full repo scan.
This happens because sparse matcher with two working parents is different from with one working parent - It switches from returning one matcher for the gitignore to returning both parent's matchers in a unionmatcher -- even if they are identical, the hash changes, triggering a full repo can.
After discussing with quark-zju, he mentioned the fsmonitor gitignore change detection has been mostly-broken for a long time ([1], [2]), and is no longer worth maintaining. The perf costs outweigh correctness and implementation of "correctness" costs, which are esoteric (you update to master with a dirty file that master happens to have unignored -- but unignores are rare) and easily fixed by the user (saving any of the files fixes it). Lastly, quark-zju's treestate stack already does this, so this effectively brings those perf benefits early.
In the longer team, the correct fix would be tracking ignored files (including those outside sparse) in treestate, and examining them.
Wrapped in a config in case it goes wrong.
[1] 6dea1701f1 (.hgignore only invallidated if the toplevel .hgignore changed, which was rare -- at Facebook usually the subdirectory ignore was changed)
[2] In `matcher.py`, the `gitignorematcher.__repr__` returns a static string.
Reviewed By: quark-zju
Differential Revision: D8337682
fbshipit-source-id: 725b3eb8fba06940fd2db0903bcecf7416e64e0d
Summary:
this is needed because `hg cloud sync` can be triggered by external serviced like scm_daemon on behalf of the user,
so it should just fail rather than expect user to type the password, so we change ui ssh option to the bgssh (background ssh) that is defined in infinitepush section
Reviewed By: markbt
Differential Revision: D8331723
fbshipit-source-id: 28f9d007702e4f6ed5216114921375b76def3f93
Summary:
In a previous diff we added logic to delete corrupt packs in the cache
directories, but the logic only caught issues after the pack was loaded. We also
need to catch issues while loading the pack. This happens when a pack file is 0
length.
Reviewed By: quark-zju
Differential Revision: D8324248
fbshipit-source-id: c2fcb4f69c4627cce316d626e91f752ef562a0d9
Summary:
An updated version of D8174246 with a test. Max memory usage is very useful to us to understand which commands are using too much memory, and the existing wrapper metrics don't work when chg is used (they measure chg's usage).
Windows support will come later as it is a bit more involved.
Reviewed By: DurhamG
Differential Revision: D8318584
fbshipit-source-id: 323450bc7ab376014d70106beb5d4fdcc7fba0c8
Summary:
In the new corruption fix strategy, we're only going to strip the
entries from the root treemanifest revlog. This will leave all the
still-referenced sub-trees available for fetching if clients still have old
references to them, while preventing new references from being taken from the
root nodes.
Reviewed By: phillco
Differential Revision: D8284287
fbshipit-source-id: 4cc20bfdbf7ae5d32074dae22da4d37717ee5728
Summary:
Use the command-level template and the `nodechanges` template keyword.
Drop the use of formatter.
`-Tjson` is not treated specially so it will just print `json`. If we'd like
to make it output `{"nodehcnages": ...}` (probably not a list with one
element), it's doable by having some mapping from `json` to a default
template.
Reviewed By: mitrandir77
Differential Revision: D8221076
fbshipit-source-id: 75575a98324d0d069789e0e915d5aa282ca0d4bb
Summary:
According to scuba chart https://fburl.com/scuba/mn5nfdna, these are
not used by any critical client, don't seem useful and can be removed.
Reviewed By: quark-zju
Differential Revision: D8232616
fbshipit-source-id: ae07cfb00bfe45cd6f8c31f3fa0871c9ba1fdadf
Summary:
It could affect pushrebase behavior - pushrebase server might have a large
set of "missing" nodes, and take that as the commits to rebase. Then fail
with "cannot rebase public commits".
Maybe there is a better way to fix it so it only affects pull but not push.
But for now it seems safer to disable it for automation.
Reviewed By: DurhamG
Differential Revision: D8049229
fbshipit-source-id: 3612e2fb2c1c825e8974156c27b03966ee745c98
Summary:
They're actively fighting against the clang-format config
and don't have an auto-fix.
Reviewed By: quark-zju
Differential Revision: D8283622
fbshipit-source-id: 2de45f50e6370a5ed14915c6ff23dc843ff14e8a
Summary:
Previously, when adding temporary pack files to the local data store,
we added them at the end. This causes problems with the recently added ondemand
generating datastores, since when we go down the line of stores we will hit the
generating store (which will create a tree from flat manifest, which is slow)
before we hit the bundle store that actually contains the data.
Let's reorder the stores so the bundle stores go at the front, since they are
likely to be used.
Reviewed By: phillco
Differential Revision: D8291415
fbshipit-source-id: 4f9799cf755b2b1f7fe8d8d1c041a3b3f2c7e463
Summary:
Optimized builds were giving a warning about shadowed variables, which
broke some of our continuous builds. Let's fix those.
Reviewed By: phillco
Differential Revision: D8287865
fbshipit-source-id: 8a2d023be93499ed6458a35ba8e8cb4d83d59e4a
Summary:
We want to better understand why people are getting "scanning working copy", so this is a first step towards that -- log every time we call `invalidate()` with the reason.
If one reason ends up dominating the others (e.g., "exception") we can drill down later.
Reviewed By: quark-zju
Differential Revision: D8272450
fbshipit-source-id: 46cb963eefa5ca60df81bdcd8f1e232855c106b7
Summary:
Previously, if a client sent a gettreepack request and specified a
non-root directory, the server would go back to the root of the tree and send
the whole tree. Now that treemanifest has a walksubdirtrees function for
iterating over a specific subset of a tree, we can make it return just the
subset.
This diff does delete the treememoizer class. This was previously used to keep
trees in memory if they would be needed again (for instance, as a base node for
a future tree). We aren't able to provide this functionality when using
walksubdirtrees, because python doesn't have access to the underlying sub
directory tree objects. This class was probably unnecessary anyway, since we
usually only request a small number of trees at once.
Reviewed By: quark-zju
Differential Revision: D8217360
fbshipit-source-id: 1db970659eabe2829acc6d49a15eeb9345ce4a5b
Summary:
In a future diff we'll be adding the ability to request just a
subsection of a tree. In order to do that, we need to expose an api that can
iterate of just a subsection of the tree.
Since the treemanifest object itself acts mostly like a dictionary, and doesn't
expose sub-tree functionality, we expose sub-tree walks as a limited use class
level function.
Reviewed By: quark-zju
Differential Revision: D8216726
fbshipit-source-id: 0d72ef041fd6a973387f975b21e55c82f7ec789c
Summary:
A recent change renamed the methods assuming they were rust methods
but these methods are used in the python code and therefore, already had the
correct naming. This broke the one of the python tests calling the methods
which will pass now.
Reviewed By: DurhamG, quark-zju
Differential Revision: D8279742
fbshipit-source-id: 01761ad033c6669a0ae54dc1beb853a0d56df27d
Summary:
just refactoring to improve the code quality
the main improvement is that I separated TcpReceiver to a different service,
any other services can register callbacks with TcpReceiver service.
For WorkspaceSubscriberService callbacks are implemented using mpsc channel to notify the main WorkspaceSubscriberService thread and single atomic flag that allows running subscriptions to join.
Another improvement is that I added logic to run cloud sync on the first keep alive after connection errors
Reviewed By: markbt
Differential Revision: D8226109
fbshipit-source-id: 3fe513da9273b28b2262948ecdf620821e7ab313
Summary:
Use the new stack-by-stack commit backup for infinitepushbackup as well as
commitcloud.
Reviewed By: DurhamG
Differential Revision: D8241380
fbshipit-source-id: 0c1481ad1ab5759ab642d0b02c7fe08523056292
Summary:
Rather than pushing a bundle containing all unpushed draft commits, instead
push stack by stack. This reduces the individual bundle size, and means
failure to push some commits means we can still make progress.
Reviewed By: DurhamG
Differential Revision: D8241383
fbshipit-source-id: 6f1528613fa228627fe156a8e4720dd46929ebe2
Summary:
Use the repo connectionpool to connect to the server. This allows sensible
re-use of connections whilst gracefully handling errors.
Reviewed By: DurhamG
Differential Revision: D8241382
fbshipit-source-id: f43669a32e4b1ff57193ea0959c57ff14526a998
Summary:
Split the pushing of commitcloud commits and infinitepush bookmarks into
separate functions, in preparation for pushing commits stack-by-stack.
Reviewed By: DurhamG
Differential Revision: D8241385
fbshipit-source-id: dc82c3470259b0ff738ebcdf6901ee8052405379
Summary:
Refactor pushing of draft heads into infinitepush; we want to call this from
infinitepushbackup, too.
Reviewed By: DurhamG
Differential Revision: D8241381
fbshipit-source-id: d5eb06efd6cd9382e016b0738d9b3e89e524de53
Summary:
Remove the cleanup from pushbackupbundle. As we now wait for the bundle
response, this is no longer necessary, and it prevents re-use of the
connection.
Reviewed By: DurhamG
Differential Revision: D8241379
fbshipit-source-id: a885198c49dce1beb1385a61a209d299ed3d2d02
Summary:
Split the infinitepush backups code into a separate module.
The infinitepush extension is responsible for providing scratch pushes, as well
as the server-side implementation for storing scratch bundles.
The infinitepushbackup extension uses this feature to provide automatic backup
support. It is an alternative to the commitcloud extension, and the two will
be mutually exclusive.
Reviewed By: DurhamG
Differential Revision: D8204418
fbshipit-source-id: 9279c426da35d451a5e7b39a72f2c91d364e3fd4
Summary:
There was a bug where if we tried to convert a flat manifest into a
tree while the flat manifest commit was being committed, the fast path
conversion would produce incorrect results because it tried to access the commit
data. If len(changelog) == 5, then changelog.revision(5) returns the nullid
data, which caused the bug.
This is absurb behavior from the changelog, so I will change that in a future
diff.
Reviewed By: phillco, singhsrb
Differential Revision: D8270372
fbshipit-source-id: 27bbfb2e54de35553b880954f06e76fe4fc0d47b
Summary:
In a future diff we'll be adding the ability to iterate over just a
subsection of a tree. In order for the resulting paths to be correct, we need to
be able to prime the start path. To do so, let's add a path argument to the
SubTreeIterator, and set it to empty string everywhere it's currently used.
Reviewed By: quark-zju
Differential Revision: D8216727
fbshipit-source-id: 1969f380f7ad13f54a7dbcd2283b0428c4480ab4
Summary:
In a future diff we'll be adding a function that also needs to turn a
python store into a ManifestFetcher. So let's move that logic to a separate
function.
Reviewed By: quark-zju
Differential Revision: D8216728
fbshipit-source-id: f6d2e9577eccfcb015797a519264bd3b241ebae5
Summary:
Now that the subtree iterator doesn't contain a reference to the
treemanifest directly (it instead holds a ManifestPtr reference), we no longer
need to persist a reference to it at the python layer.
Reviewed By: quark-zju
Differential Revision: D8206301
fbshipit-source-id: 531d4399f0f54a47e0eb741f419e6242188a83ad
Summary:
Now that markPermanent no longer requires a ManifestEntry to update the
node, the resultEntry field on all the iterators is unused. Let's delete it.
Reviewed By: quark-zju
Differential Revision: D8156808
fbshipit-source-id: 21aeb050c91552c7e8973a33dd7e347457125a64
Summary:
Now that the node field is accessed via a getter that can lazily
compute it, let's get rid of the logic that proactively sets it. This will make
it easier to refactor the subtree iterator in a later diff.
Reviewed By: quark-zju
Differential Revision: D8156809
fbshipit-source-id: aabc3f68cc41baea0c166b9a2bd68bea6b1a6a03
Summary:
Previously the ManifestEntry node field was public and consumers could
read it directly. This required that the node field be updated manually, which
added complications to other code paths. In a future diff we'll be simplifying
the SubTreeIterator code to not require setting node, and to do so we make node
calculated in this accessor.
Reviewed By: quark-zju
Differential Revision: D8156807
fbshipit-source-id: e1908b2149bc0ec8fb1279e37b8a66988ff125c6
Summary:
This switches the treemanifest native iterators to use the ManifestPtr
class instead of Manifest*. ManifestPtr does ref counting, and in a future diff
we'll use this ability to have the subtree iterator iterate over a portion of a
larger tree so we can serve just parts of trees to gettreepack requests.
Reviewed By: quark-zju
Differential Revision: D8156806
fbshipit-source-id: 4dbb60d008ac8d0c789c3f2db8f7e567c5869539
Summary: This is just a refactor to address the naming scheme.
Reviewed By: quark-zju
Differential Revision: D8269217
fbshipit-source-id: 8c52d2c67837550e0b7dc1a45b3faf9a80319b61
Summary:
This adds a command that can be used to fill a database with data from
existing revlogs. This will be useful in the future for uploading new
repositories as well as fixing any database corruption.
In particular, this is important as a precaution to fixing the treemanifest
corruption.
Reviewed By: farnz
Differential Revision: D8108361
fbshipit-source-id: 20717c2057f2bed82c98323c4a3e9d0c6dbd7df5
Summary:
1. Enabled a number of additional C++ compiler warnings in Eden.
2. Fixed warnings-turned-errors that resulted from this change.
Reviewed By: simpkins
Differential Revision: D8132543
fbshipit-source-id: 2290ffaaab55024d582e29201a1bcaa1152e6b3e
Summary:
We were encountering issues where readonly connections where failing
because the serve was returning a Permission denied error while fetching trees.
It turns out the ondemand tree generation was attempting to open a transaction
and create trees on the server. Since the server should have all the trees
anyway, we shouldn't do this ondemand conversion.
We still need on demand conversions for infinitepush bundles, so we'll leave
that in.
Reviewed By: singhsrb
Differential Revision: D8226459
fbshipit-source-id: 2d05faefbe3ae92ccb81072e012673f29c9cf0d9
Summary: Sometimes it's useful to be able to resume generation from a commit that isn't the tip (e.g., after recloning or pulling).
Reviewed By: singhsrb
Differential Revision: D8217644
fbshipit-source-id: fd622ed20e1e3782f3bc3e2df58a2d33425ea496
Summary:
I'd like to make it so that eden clones don't need to
rebuild the svn metadata on the first operation after a clone,
and don't need special post-clone logic like `hg-new-workdir` has.
Reviewed By: quark-zju
Differential Revision: D8219138
fbshipit-source-id: da1e9c9a1389893ecd49962ad3143445e85fb247
Summary:
btrfs has a bug involving truncate not returning the correct error
code. We already handle it in other cases, so let's extend it to these uses of
truncate. Hopefully the need for this disappears soon.
Reviewed By: quark-zju
Differential Revision: D8152297
fbshipit-source-id: f55602ff5e0ec36346c547bfd4b6d0f6e4127500
Summary:
Previously we weren't including p2 when creating trees, this resulted
in incorrect conversions.
Reviewed By: phillco
Differential Revision: D8152253
fbshipit-source-id: c4f8c79b40532c5162b15032962fbc6a78d44b5f
Summary:
The memcmp contract is that the result is >0, 0, or <0. If <0 it's not
guaranteed to be -1, so let's broaden the check a bit.
Reviewed By: phillco
Differential Revision: D8190843
fbshipit-source-id: 78b864a639851a041c1a393f21a979d13eaa2e9c
Summary:
This switch converting flat manifests to tree manifests to use a fast
path where we avoid decompressing the whole manifest whenever possible. This
will speed up tree conversions tremendously.
Reviewed By: phillco
Differential Revision: D8130357
fbshipit-source-id: aa91f9445216d2c5cb60d51c21ae0f6636359d9c
Summary: The recent reformat caused a few test failures, this should fix them.
Differential Revision: D8207359
fbshipit-source-id: ced37aff3038715e5876fe27b58a60ebe45e4097
Summary: Mostly empty lines removed and added. A few bugfixes on excessive line splitting.
Reviewed By: quark-zju
Differential Revision: D8199128
fbshipit-source-id: 90c1616061bfd7cfbba0b75f03f89683340374d5
Summary:
This makes the signature match _getfastflatdiff, so we can unify them
in the next diff.
Reviewed By: phillco
Differential Revision: D8130358
fbshipit-source-id: 4bc51b79d0432b3347dfd7996c9f586f0c65999f
Summary:
This refactors the new tree creation logic from a single function to
two functions, one that computes the differences in (added, removed) format, and
another that accepts adds/removes and applies them to a copy of the parent tree.
This will help in a future diff where we want to unify code paths to use the
fast diff path.
Reviewed By: phillco
Differential Revision: D8122900
fbshipit-source-id: 82699dc4bbf69dd565f8564e71f8a0726659313e
Summary:
We already had logic that could compute a new tree from a manifest node
quickly (using the manifest revdiff function to avoid reading entire manifests).
Let's refactor that to a function so in a future diff we can unify it with
_getflatdiff so all paths can take advantage of the performance.
Reviewed By: phillco
Differential Revision: D8122469
fbshipit-source-id: 5cb315327356cece9b2fed3e2c57088179ef838e
Summary:
A future diff will optimize the performance of this particular
function.
Reviewed By: phillco
Differential Revision: D8121843
fbshipit-source-id: 22187aec658cef7b2f724fae113a651c24d79851
Summary:
A future patch will improve the performance of convert flat manifests
to tree manifests by relying on the stored manifest delta instead of hydrating
the whole manifest. To do this, we need to change _getnewtree to not depend on
taking the entire flat manifests. So let's refactor it to take a diff instead,
and in a future patch we will optimize how we compute that diff.
Reviewed By: phillco
Differential Revision: D8121627
fbshipit-source-id: 41bbba6e98cd605ee185618b5f99883a9e0fd7cf
Summary: Added const and override where appropriate to some classes in scm.
Reviewed By: aary
Differential Revision: D8176501
fbshipit-source-id: a9247aee569692973a822d002e951dad42c42be6
Summary:
We're seeing "Permission denied" errors on some of our automation, but
it's difficult to track down where it's coming from. Let's make each message
more descriptive.
Also updates, hgsql to handle the hg-rsh hook, which I noticed while
investigating.
Reviewed By: phillco, farnz
Differential Revision: D8188414
fbshipit-source-id: 5f8c99e8ba896c2636b1a04716125bc6a9df0591
Summary:
These are secret options for SCM Daemon
1. specify the version that comes from notification.
if notification comes from itself, we can save server time from unnecessary requests.
2. respect auto backup current settings
before auto backup setting were only respected by the background backup triggered when you make a commit. It is logical that SCM Daemon will also respect that settings
Reviewed By: markbt
Differential Revision: D8143296
fbshipit-source-id: 0d407d489a723543bdb2ce25af8b9fd6687e2d53
Summary:
Going to make changes to `mercurial/` for cleaner fsmonitor support
directly. So let's move the Rust python bridge there first.
Reviewed By: markbt
Differential Revision: D7909174
fbshipit-source-id: 454d784b5dca18a3af9328fc7b2f342cd4188cf6
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
Summary:
This adds logic to catch unknown exceptions from inside certain pack
files and delete them as a way to remediate issues.
Reviewed By: phillco
Differential Revision: D8129341
fbshipit-source-id: f3d8d414de7eb4a612306d579e643c2094ca93fc
Summary:
This refactors the common pattern of looping over a series of pack
files performing an operation to a separate function. This will allow us in a
later diff to have logic that detects bad pack files and removes them from the
list.
Reviewed By: singhsrb
Differential Revision: D8127873
fbshipit-source-id: 0ab20833a8daedbcfb9c030e66c6ba004dec33f9
Summary:
If tokens are stored with whitespace around them, sending them as OAuth headers
may fail. Make sure we strip them.
Reviewed By: mitrandir77
Differential Revision: D8160309
fbshipit-source-id: b403be6b11f25087ebc1ecdbe9a58b7f9dbda2f8
Summary:
Make it show "added", "removed", "changed" so it looks similar to the
original commit template.
Reviewed By: singhsrb
Differential Revision: D8158006
fbshipit-source-id: 1a1578bd263e7870c27e7169537e808ffd8add9f
Summary: This allows the function to optionally take arguments.
Reviewed By: singhsrb
Differential Revision: D8158007
fbshipit-source-id: a1eb89a00694c69d8d76dc4e2f7c0ce99164bcfb
Summary:
Make the (user-facing) log for prefetching trees say how many commits (or which
commit) it is prefetching for.
When remotefilelog.debug is enabled, print out the manifest identity when
fetching a tree during normal operation.
Reviewed By: quark-zju
Differential Revision: D8056555
fbshipit-source-id: dc0c7bc5c949a0674d5f553661e736bc545134b4
Summary:
In previous diff we've added the ability to log wireproto requests.
However getfiles wireproto request is special, because it sends arguments in a different way.
Reviewed By: farnz
Differential Revision: D8097782
fbshipit-source-id: f598acd155e842e90d14a295fc71b1886e5a7fb0
Summary:
The querystring is encrypted with SSL
However, URLs can be stored in web server logs - typically the whole URL of each request is
stored in a server log. This means that any sensitive data in the URL (e.g. a
password) is being saved in clear text on the server
Reviewed By: farnz
Differential Revision: D8075914
fbshipit-source-id: 4eadd362755272414dadad069e7ba04db7a01ee8
Summary:
For the smartdate template function, when displaying commits in the last week
using the day name, we compare the exact times. This means commits made late
in the day on one day, will still show using the day name early in the day the
following week.
Instead, compare just the dates. We can simplify the other date-based
comparisons, too.
Reviewed By: farnz
Differential Revision: D8113953
fbshipit-source-id: b9bed59ccf349e7274ad65d7ed5a7e08c50d4b53
Summary:
The method looks like a foot-gun, and it's O(all entries) instead of
O(cached entries). Change `get_tracked_filtered_key` to take an identity of
the filter function explicitly to solve the problem.
Reviewed By: markbt
Differential Revision: D7861765
fbshipit-source-id: a57ca4a7597120a5b00c63f3f373a62e19e5a834
Summary:
Improve `makedirs` error message so it would be more helpful when there are
broken symlinks.
Differential Revision: D8108794
fbshipit-source-id: 08013022642efde946ef9d5c6b06b4763f4ad68f
