Summary:
Importing an extension should be side-effect free. Move the patching to uisetup
so they only get executed when the extension is enabled.
Reviewed By: kulshrax
Differential Revision: D14694622
fbshipit-source-id: c2802e0f4d852445dd7900f11d4ede6e58883d94
Summary:
Change the index key from decimal to binary to make the index more compact and
efficient. Move the binary to the beginning to avoid common prefixes and speed
up lookups.
Reviewed By: singhsrb
Differential Revision: D14694623
fbshipit-source-id: e908d287c1d3559639dbafa39a465f89b8f6f14e
Summary: This makes the progress visible.
Reviewed By: singhsrb
Differential Revision: D14694621
fbshipit-source-id: bb80546cb513c423471ba004bf00db7431046bcf
Summary:
Use revision numbers to make sure the index contains all commits.
Also include svnrev in the same index. This has multiple benefits:
- Handles svnrev mapping, too. Easier to drop hgsubversion.
- Handles the special commit without globalrev or svnrev gracefully.
It takes about 3 minutes to build the index for www.
I removed code about phases, or heads. The code runs on hg servers
and hg servers (hgsql) have hard dependency on rev numbers.
If we get a chance to migrate away from client-side local changelog,
we'll have Mononoke equivalent commit mapping by that time (or, push
commit mapping work to other teams).
Reviewed By: singhsrb
Differential Revision: D14694620
fbshipit-source-id: 9b74782f4c37b6620ab7168f7c7199d0cc5bc396
Summary:
This is a requirement to enable Phabricator migration to a globalrev
based repository. The release engineering team has also expressed interest in a
faster way to lookup the commits using globalrev.
I am just using indexed log for now to enable faster lookup. We might not need
to backfill the indexed log since we mostly care for the newer commits. But if
we do need to, we can do it later.
Reviewed By: quark-zju
Differential Revision: D14687707
fbshipit-source-id: fe02e9a72b9edb57cfd49a294642014c7204c058
Summary:
This is required for the fast commit lookup based on globalrev in
D14687707. This code is heavily inspired from DurhamG's implementation for the
git-hg mapping in the `hggit` extension.
quark-zju has good suggestions for improving this to include the svnrev as well
which can eliminate the requirement for a svnrev-hg mapping in the future. We
can certainly do those in a later iteration.
Reviewed By: quark-zju
Differential Revision: D14674111
fbshipit-source-id: d09d9e65ac0abcafd478c873086898f787398867
Summary:
This diff fixes the behavior of `skip_subtree` when called on a Leaf. The bug is
that the path is not correctly handled in this case. The name of the file
continues to stay in the path resulting in incorrect path names for all
subsequent calls to `path()`.
The high level perspective is that `skip_subtree` is a no-op in a Leaf node.
To fix, clarify the behavior and improve readability of the code we ad a new
state that handles poping elements from the path.
Durham noticed this bug when reviewing D14347655.
Reviewed By: quark-zju
Differential Revision: D14654557
fbshipit-source-id: 625278366e492a3048dddc44f9234a06d6928b7e
Summary: This is just for convenience and making the code cleaner.
Reviewed By: quark-zju
Differential Revision: D14689470
fbshipit-source-id: 911f49e24a3bb504098537169e9096b080a86535
Summary:
Allow running `hg push --to scratch/bookmarkname -r X` where X is a public
commit.
Note that updating a scratch bookmark that is on a public commit will
require a force push.
Reviewed By: quark-zju
Differential Revision: D14527730
fbshipit-source-id: dbc234b43d74a36b7403bea19e1e917ad69cdbbf
Summary:
It's hard to clone a `Fn`. But `fn` can be cloned. Change the API to use `fn`
instead.
Cloning `IndexDef` allows the same index definition to be used by multiple
Logs. It's used by upcoming diffs.
Differential Revision: D14688181
fbshipit-source-id: 6fda03a5f744dc90ee5d7ad3f36c243602f33510
Summary:
Before this change, the following command will redraw the progress bar
frequently:
./hg debugprogress --sleep 1 100000 --config progress.refresh=1 --nested
After this change, the above command only redraws once per second.
This should reduce overhead in `repack.py:repacker.repackdata`.
Differential Revision: D14693924
fbshipit-source-id: baa281ba2e262d2ef8e83c2460f7edf5400db1af
Summary: This makes it easier to debug real world progress related issues.
Differential Revision: D14691592
fbshipit-source-id: 2a653f657b6ba9f9369f1551845949ac7e4d0b9d
Summary: Previously the debug commands for the Eden API did not check whether HTTP data fetching was enabled before trying to access the client singleton, which would result in a crash if HTTP was disabled. This diff adds explicit checks so that the commands will abort instead of crashing.
Reviewed By: quark-zju
Differential Revision: D14692893
fbshipit-source-id: d36e241d8460dadb555a15c92aca9334ba00f34c
Summary:
When doing a pushrebase, the server needs to mount a bundle over the
exist repository to run hooks. With lfs enabled this uses changegroupv3, and
with a treeonly client no manifests are sent as part of the changegroup (they
are instead sent as a separate part).
The combination of these three things hits an edge cases where if there are no
manifests in a changegroupv3 bundle, the bundlerepo class may not skip the
entire manifest part, and therefore the bundle appears to have no files in it,
even though the data is there.
This only happens if the server is in treeonly mode, or during pushrebase hooks
where we fake treeonly mode briefly. Since we don't have any server repos that
are treeonly and lfs yet, this didn't break us, but it would have. We caught it
because some of our hooks are affected when they try to read file contents.
The fix is to account for the cgv3 case in the bundle manifest consuming code.
Eventually we can get rid of all of this logic once we get rid of revlogs and
changegroups.
Differential Revision: D14676674
fbshipit-source-id: 86ac1ff4a92e37c4da3767a66760b55b658c836f
Summary:
When a server is treeonly and using lfs, pushrebase pushes can fail
because bundlerepo doesn't know how to process a changegroupv3 with an empty
manifest section. This diff adds a test demonstrating the failure.
A future diff fixes it, and explains the bug in more detail.
Differential Revision: D14676673
fbshipit-source-id: fb3c4815a42e69e31b4f1673797de95516a83406
Summary:
With `remotefilelog.fetchpacks`, some pathological cases would end up creating
tons of packfiles and we would thus spend a significant amount of time in
refresh, scanning for new packfiles. Jeremy noted that for every refresh,
stat(2) would be ran on every file, even though only a handful of packfile
would be added.
Differential Revision: D14674093
fbshipit-source-id: df198d208e3f4e1d667e7bdd069a793984683282
Summary:
Now that repacks runs more often, it's easier to trigger a race between repack
deleting packfiles, and another mercurial process listing the packfiles and
trying to open them. If the packfiles are deleted after the directory listing,
all the packfiles will fail to be opened and were mis-reported as corrupted.
Reviewed By: quark-zju
Differential Revision: D14648308
fbshipit-source-id: c3b852f669e28db6f622bde217f339533e094223
Summary:
On Windows, SIGPIPE isn't present in the signal module. This caused the exit
code to raise an exception when trying to install a handler for it.
Reviewed By: DurhamG, strager
Differential Revision: D14655142
fbshipit-source-id: 68ddfc5250a4b706c5bc6a7ad1add37e1ccbe26e
Summary:
This makes it easier to compare benchmark results between abstractions.
A sample of the result is listed below. Comparing to radixbuf, which is highly
optimized and less flexible, indexedlog is about 10x slower on insertion, and
about 3x slower on lookup.
indexedlog:
index insertion (owned key) 90.201 ms
index insertion (referred key) 81.567 ms
index flush 50.285 ms
index lookup (memory) 25.201 ms
index lookup (disk, no verify) 31.325 ms
index lookup (disk, verified) 46.893 ms
log insertion 18.421 ms
log insertion (no checksum) 12.106 ms
log insertion with index 110.143 ms
log flush 8.783 ms
log iteration (memory) 6.444 ms
log iteration (disk) 6.719 ms
raidxbuf:
index insertion 11.874 ms
index lookup 8.495 ms
Differential Revision: D14635330
fbshipit-source-id: 28b3f33b87f4e882cb3839c37a2a11b8ac80d3e9
Summary:
This is just a trivial test case showing the overhead of xxhash.
log insertion 18.359 ms
log insertion (no checksum) 7.835 ms
Differential Revision: D14635329
fbshipit-source-id: adc2629c0c41aaab48d29d467849e4d96eb01c51
Summary: Change the code to work with newer `rand` crate.
Reviewed By: kulshrax
Differential Revision: D14635328
fbshipit-source-id: 007f6749f2eab781a7dcf7d49b19aff1c81089b4
Summary: Add a function that adds all of the entries from a given iterator into a HistoryPack.
Differential Revision: D14647767
fbshipit-source-id: 29a71b37da86125e14135c40c279bfc8a454b568
Summary:
Using cargo for development using mercurial will result in use of random target
directories depending on where cargo is invoked from.
This diff sets the target directory for cargo in the on disk config file so
that all invocations of cargo from our project reuse the previously built
artifacts.
This may make it more problematic to build for multiple targets because cargo
does not recompile when flags change.
Reviewed By: quark-zju
Differential Revision: D14606468
fbshipit-source-id: 56c0b3b192ee94e1ddafe0949cc2267bb995a696
Summary:
Add basic support for `undo` by updating visibility according to what the undo
extension would normally do with obsmarkers.
A future enhancement would be to use the `draftheads` directly as the new
visibleheads, however:
* `undo` currently uses `heads(draft())` which doesn't take into account
secret commits.
* In order to support the `--branch` flag, we would need to make significant
changes. The flag is marked as ADVANCED and isn't used very much, so we may
consider removing it.
Reviewed By: quark-zju
Differential Revision: D14603196
fbshipit-source-id: 5b8c91b4d8fa2b28cf9ad5726d88a3735aca50ad
Summary:
When changes are undone so that an earlier view of the repo is present, fate
should only be calculated for commits that are obsolete in the current view.
This makes smartlog look correct after `undo` commands have been used.
Reviewed By: quark-zju
Differential Revision: D14603198
fbshipit-source-id: 3adaad94b9fe80e8a622d469513c784b772ee235
Summary:
Record mutation entries inside transactions. This ensures that they only get
written when the transaction completes.
Indexedlog doesn't have support for pending writes, so mutation entries are not
visible to hook scripts.
Reviewed By: quark-zju
Differential Revision: D14566781
fbshipit-source-id: eb4c7bbd3878df82e8e7096a69509525f9fb93c0
Summary:
Remove the last parts of the mutationcache, relying directly on the indexedlog
indexes.
The set of obsolete commits is now computed on the fly - we will cache this
later.
Reviewed By: quark-zju
Differential Revision: D14566780
fbshipit-source-id: 770d0d9d507bed982e88518dcf63a4c2d6b46a69
Summary:
When visibility changes, the cache of filtered revs is no longer valid and must
be cleared.
Reviewed By: quark-zju
Differential Revision: D14566777
fbshipit-source-id: 681a73c126f3258baa7f61157aba557ea8aceeca
Summary:
To prepare for transaction support, we need to move the store out of the cache
object and into its own property on the localrepo.
Reviewed By: quark-zju
Differential Revision: D14566776
fbshipit-source-id: 8fff03a86953fb60ed06dbbdcdd0bffc379bd047
Summary:
Computing mutation entries for all local commits is expensive when there are
lots of local draft commits. Ideally we would have an indexed changelog that
would make these lookups fast, but until we have that, put entries in the
mutation store for these commits to take advantage of the fast lookup there.
Reviewed By: quark-zju
Differential Revision: D14566782
fbshipit-source-id: cc3a05715337a510a65d8ff436c59d16d0f0447e
Summary:
The computations for instabilities revsets (`orphan`, `contentdivergent` and
`phasedivergent`) are complex, and aren't really necessary for the way we use
mutation information.
The `orphan` revset can be implemented as an alias for `obsolete():: -
obsolete()`. The other conditions can be detected on a case-by-case basis
later if we need them.
Reviewed By: quark-zju
Differential Revision: D14566784
fbshipit-source-id: 60b8443ad4c0c82d8250d8e9a10e73fae770daa8
Summary:
To support future alternate hashing schemes, predecessor identities should be
prefixed with the hashing scheme in use.
Currently there is only one hashing scheme: `hg`, which is the Mercurial
hashing scheme.
Reviewed By: quark-zju
Differential Revision: D14566778
fbshipit-source-id: baaaf2f078886a1cc7ac20d12923a63b4da09db6
Summary: `std::fs` is only needed for Windows. Do not "use" it on *nix systems.
Reviewed By: sfilipco
Differential Revision: D14634779
fbshipit-source-id: 9fd9a29ae27e13f00b4adbc83a74bd92a1b1658c
Summary:
Change fields in IndexDef to private. Provide a public constructor method and
switch users to use that instead. This makes it possible to change the IndexDef
struct in the future (ex. having extra optional fields about whether the index
is backed by radix tree or something different).
Differential Revision: D14608955
fbshipit-source-id: 62a413268d97ba96b2c4efd2ce67cd4fa0ff4293
Summary: Update Rust toolchain to 1.33.0 with fixes to make our code compatible with 1.33.0.
Reviewed By: Imxset21, kulshrax
Differential Revision: D14608312
fbshipit-source-id: 2d9cf7d01692abaed32f9adffa0e5eb51cfacb4f
Summary:
The upstream has removed it in https://phab.mercurial-scm.org/D2393. Do the
same.
The deleted C++ code seems to leak `Py_False` if `usemanifestv2` is not set.
Reviewed By: singhsrb
Differential Revision: D14611525
fbshipit-source-id: d828526c31aaa861d100a846bba79d1f5898e245
Summary:
A user hit cryptic error message:
File "revset.py", line 423, in ancestor
ancestors = repo.changelog.index.ancestors
AttributeError: 'list' object has no attribute 'ancestors'
That was because they have run `rm -rf` and the `.hg` directory was mostly
removed. `hg` falls back to revlog v0, and the code in question obviously
dislikes revlog v0.
Change the error message to just say repo corruption.
Since there are code depending on revlog v1, and we don't use v0 or v2. Remove
both of them.
Reviewed By: singhsrb
Differential Revision: D14610769
fbshipit-source-id: 9015b74989d8fa4ad50bc36ec48bca26a1faa361
Summary:
Callers may call `remotenames.activepath` with the remote as a string URL. For
remotenames we must normalize these paths as we do for remotes that are passed
in as objects, for the same reason as in D14051479.
Differential Revision: D14621090
fbshipit-source-id: 8a02e9554039ad3daf3093001889cf1746ce395a
Summary:
The extdiff extension uses archive on the workingctx to make snapshots
of certain files. Prefetching fails on this because it sees placeholder file
nodes. Let's just not prefetch in this case at all.
Differential Revision: D14609691
fbshipit-source-id: 2d06bf015a8b10afa16b3b0bcb266afb102d63b9
Summary:
The configparser tests `hg::tests::test_basic_hgplain` and
`hg::tests::test_hgplainexcept` set different values for `HGPLAIN` and
`HGPLAINEXCEPT`. Since the tests run in parallel and use the same environment,
one of the tests may fail if they run at the same time.
For these tests, create a mutex for the environment and lock it for the
duration of the test, ensuring these tests do not interfere with each other.
Reviewed By: jsgf
Differential Revision: D14615394
fbshipit-source-id: 9f123668d93223655514db2ae34b05354a6b578c
Summary:
Now that we have perftracing infra, let's trace a bunch of likely
problem spots.
Reviewed By: sfilipco
Differential Revision: D14426367
fbshipit-source-id: 354a241aa9ac5d75d34062a9838d581b4f46746f
Summary:
Most of the repacks are background repacks, and most of the complaints are
coming from laptops users. Thanks to the rust repack, most of time during
repack is now spent in repacking loosefiles. While repacking them is expensive,
testing whether data is in a loosefile and obtaining it is actually pretty
fast. Packfiles have the opposite issue, repacking them is fast, but finding
data in them may be expensive if a lot of them are present.
Based on this, it makes sense to repack packfiles more frequently than
loosefiles. At first, the newly added option will be used to turn-off loosefile
repack for laptop users. A less frequent loosefile repack will be implemented
in a later patch.
Reviewed By: DurhamG
Differential Revision: D14586986
fbshipit-source-id: 5bc5c839cf8d2d78bcc4ffa016bbe3cf1b2ef3f7
Summary:
I'm not sure which change broke the test. It requires `lfs-test-server` to run,
which is not installed by default. So that's why people didn't realize it.
Differential Revision: D14607695
fbshipit-source-id: 3b0e6d4250f476abced414c00bddfe7d29d062d4
Summary:
See D14606986 for context. `repo.close` should be protected by not crashing on
stdout or stderr write errors (SIGPIPE or EPIPE). Move `deferred` to `atexit`
handler to get the protection.
Test changes are caused by ordering changes.
Differential Revision: D14607407
fbshipit-source-id: 5a42aefcec395f48b8ecb67426429ef2e41f5666
Summary:
This is the first diff of an alternative fix of D14528603. See also D14603974
for context.
We'd like atexit handlers to not crash with EPIPE or SIGPIPE when writing to
stderr or stdout. Therefore disable SIGPIPE signal handler and patch ui methods
to do so.
Differential Revision: D14607159
fbshipit-source-id: 274c5174813d402a7e0b8b5be7c8fcb0524fcdb3
Summary:
akushner and I both hit an issue where `hg log` hangs for a long time after
exiting the pager by pressing `q`. That was caused by D14528603. Revert it to
fix it.
The root cause is apparently a bug in Python 2 (or C stdlib?):
# a.py
import signal, os
signal.signal(signal.SIGPIPE, signal.SIG_IGN)
out = os.fdopen(1, 'wb', 1)
while True:
out.write(b"a\n")
# shell
python2 a.py | head -n 1
# hangs
Python 3 does not have the issue.
Reviewed By: singhsrb
Differential Revision: D14603974
fbshipit-source-id: 3b4d716133853632e56dfeb59b1858f926a2c146
Summary:
Now that we're recording perftraces, let's log their results to the
blackbox if the command exceeds a configurable threshold.
Reviewed By: sfilipco
Differential Revision: D14426366
fbshipit-source-id: 24f261aaa2b089dbc959d709e51de1c0359d976d
