Summary:
The phabricator team is trying to eliminate `DifferentialRevision` in favor of `PhabricatorDiff`
See context here
https://our.intern.facebook.com/intern/wiki/Dev_Tools/Phabricator/GraphQLObjectRename/
I'm using aliases to reduce the number of changes needed (the object from the graphql query will have the same keys)
Reviewed By: quark-zju
Differential Revision: D14703867
fbshipit-source-id: 32204315dc154db7cbff68ee286579c0200f57b2
Summary:
Loosefiles are quite slow to repack, and this is significantly affecting
laptops as the repack can potentially run for hours when a large amount of
loosefiles are present.
One of the reason for that is that an incremental repack really isn't
incremental for loosefiles as all of them are repacked at the same time.
Instead let's repack only one of the top-level directory at a time.
Reviewed By: quark-zju
Differential Revision: D14690348
fbshipit-source-id: ffba49840302ae0d99e32db410647e83e213fe64
Summary:
Currently, if you lookup a globalrev which does not exist in the
repository, it leads to a linear scan of the entire changelog even while using
the globalrev based fast commit lookup. This commit speeds up the same by
relying on the indexing information.
Differential Revision: D14726937
fbshipit-source-id: 076ad8608aa656075dcb2c1287502ffe93bed851
Summary:
This is just pending feedback from D14694620 which makes the class
interface more cleaner.
Reviewed By: quark-zju
Differential Revision: D14718748
fbshipit-source-id: 032fc25d7e5922aac5f4195eb007bd009822cb86
Summary:
Rather than having a `Builder` struct that knows how to build just one kind of Eden API client, let's have a common `Config` type that can be potentially passed to the constructor of several different client implementations.
This will allow the same config code to be re-used across different client types, as seen later in this stack.
Differential Revision: D14656757
fbshipit-source-id: 883ffd2dc0302ebe08960f079c113e2d0da2d2ca
Summary:
When the sql_repo_lock.py hook is run, the sql connection won't have been
established yet. Let's open a connection and clean it up afterwards.
Reviewed By: quark-zju
Differential Revision: D14708851
fbshipit-source-id: f20b6dacdcb12cee839163325164d2e75816100c
Summary:
The type for globalrev during lookup should be an integer instead of a
string after D14694623.
Reviewed By: quark-zju
Differential Revision: D14718749
fbshipit-source-id: eddc49ea0faa61df6ad435db2192fba4f61b0075
Summary:
Importing an extension should be side-effect free. Move the patching to uisetup
so they only get executed when the extension is enabled.
Reviewed By: kulshrax
Differential Revision: D14694622
fbshipit-source-id: c2802e0f4d852445dd7900f11d4ede6e58883d94
Summary:
Change the index key from decimal to binary to make the index more compact and
efficient. Move the binary to the beginning to avoid common prefixes and speed
up lookups.
Reviewed By: singhsrb
Differential Revision: D14694623
fbshipit-source-id: e908d287c1d3559639dbafa39a465f89b8f6f14e
Summary: This makes the progress visible.
Reviewed By: singhsrb
Differential Revision: D14694621
fbshipit-source-id: bb80546cb513c423471ba004bf00db7431046bcf
Summary:
Use revision numbers to make sure the index contains all commits.
Also include svnrev in the same index. This has multiple benefits:
- Handles svnrev mapping, too. Easier to drop hgsubversion.
- Handles the special commit without globalrev or svnrev gracefully.
It takes about 3 minutes to build the index for www.
I removed code about phases, or heads. The code runs on hg servers
and hg servers (hgsql) have hard dependency on rev numbers.
If we get a chance to migrate away from client-side local changelog,
we'll have Mononoke equivalent commit mapping by that time (or, push
commit mapping work to other teams).
Reviewed By: singhsrb
Differential Revision: D14694620
fbshipit-source-id: 9b74782f4c37b6620ab7168f7c7199d0cc5bc396
Summary:
This is a requirement to enable Phabricator migration to a globalrev
based repository. The release engineering team has also expressed interest in a
faster way to lookup the commits using globalrev.
I am just using indexed log for now to enable faster lookup. We might not need
to backfill the indexed log since we mostly care for the newer commits. But if
we do need to, we can do it later.
Reviewed By: quark-zju
Differential Revision: D14687707
fbshipit-source-id: fe02e9a72b9edb57cfd49a294642014c7204c058
Summary:
This is required for the fast commit lookup based on globalrev in
D14687707. This code is heavily inspired from DurhamG's implementation for the
git-hg mapping in the `hggit` extension.
quark-zju has good suggestions for improving this to include the svnrev as well
which can eliminate the requirement for a svnrev-hg mapping in the future. We
can certainly do those in a later iteration.
Reviewed By: quark-zju
Differential Revision: D14674111
fbshipit-source-id: d09d9e65ac0abcafd478c873086898f787398867
Summary: This is just for convenience and making the code cleaner.
Reviewed By: quark-zju
Differential Revision: D14689470
fbshipit-source-id: 911f49e24a3bb504098537169e9096b080a86535
Summary:
Allow running `hg push --to scratch/bookmarkname -r X` where X is a public
commit.
Note that updating a scratch bookmark that is on a public commit will
require a force push.
Reviewed By: quark-zju
Differential Revision: D14527730
fbshipit-source-id: dbc234b43d74a36b7403bea19e1e917ad69cdbbf
Summary:
Before this change, the following command will redraw the progress bar
frequently:
./hg debugprogress --sleep 1 100000 --config progress.refresh=1 --nested
After this change, the above command only redraws once per second.
This should reduce overhead in `repack.py:repacker.repackdata`.
Differential Revision: D14693924
fbshipit-source-id: baa281ba2e262d2ef8e83c2460f7edf5400db1af
Summary: This makes it easier to debug real world progress related issues.
Differential Revision: D14691592
fbshipit-source-id: 2a653f657b6ba9f9369f1551845949ac7e4d0b9d
Summary: Previously the debug commands for the Eden API did not check whether HTTP data fetching was enabled before trying to access the client singleton, which would result in a crash if HTTP was disabled. This diff adds explicit checks so that the commands will abort instead of crashing.
Reviewed By: quark-zju
Differential Revision: D14692893
fbshipit-source-id: d36e241d8460dadb555a15c92aca9334ba00f34c
Summary:
When doing a pushrebase, the server needs to mount a bundle over the
exist repository to run hooks. With lfs enabled this uses changegroupv3, and
with a treeonly client no manifests are sent as part of the changegroup (they
are instead sent as a separate part).
The combination of these three things hits an edge cases where if there are no
manifests in a changegroupv3 bundle, the bundlerepo class may not skip the
entire manifest part, and therefore the bundle appears to have no files in it,
even though the data is there.
This only happens if the server is in treeonly mode, or during pushrebase hooks
where we fake treeonly mode briefly. Since we don't have any server repos that
are treeonly and lfs yet, this didn't break us, but it would have. We caught it
because some of our hooks are affected when they try to read file contents.
The fix is to account for the cgv3 case in the bundle manifest consuming code.
Eventually we can get rid of all of this logic once we get rid of revlogs and
changegroups.
Differential Revision: D14676674
fbshipit-source-id: 86ac1ff4a92e37c4da3767a66760b55b658c836f
Summary:
With `remotefilelog.fetchpacks`, some pathological cases would end up creating
tons of packfiles and we would thus spend a significant amount of time in
refresh, scanning for new packfiles. Jeremy noted that for every refresh,
stat(2) would be ran on every file, even though only a handful of packfile
would be added.
Differential Revision: D14674093
fbshipit-source-id: df198d208e3f4e1d667e7bdd069a793984683282
Summary:
Now that repacks runs more often, it's easier to trigger a race between repack
deleting packfiles, and another mercurial process listing the packfiles and
trying to open them. If the packfiles are deleted after the directory listing,
all the packfiles will fail to be opened and were mis-reported as corrupted.
Reviewed By: quark-zju
Differential Revision: D14648308
fbshipit-source-id: c3b852f669e28db6f622bde217f339533e094223
Summary:
On Windows, SIGPIPE isn't present in the signal module. This caused the exit
code to raise an exception when trying to install a handler for it.
Reviewed By: DurhamG, strager
Differential Revision: D14655142
fbshipit-source-id: 68ddfc5250a4b706c5bc6a7ad1add37e1ccbe26e
Summary:
Add basic support for `undo` by updating visibility according to what the undo
extension would normally do with obsmarkers.
A future enhancement would be to use the `draftheads` directly as the new
visibleheads, however:
* `undo` currently uses `heads(draft())` which doesn't take into account
secret commits.
* In order to support the `--branch` flag, we would need to make significant
changes. The flag is marked as ADVANCED and isn't used very much, so we may
consider removing it.
Reviewed By: quark-zju
Differential Revision: D14603196
fbshipit-source-id: 5b8c91b4d8fa2b28cf9ad5726d88a3735aca50ad
Summary:
When changes are undone so that an earlier view of the repo is present, fate
should only be calculated for commits that are obsolete in the current view.
This makes smartlog look correct after `undo` commands have been used.
Reviewed By: quark-zju
Differential Revision: D14603198
fbshipit-source-id: 3adaad94b9fe80e8a622d469513c784b772ee235
Summary:
Record mutation entries inside transactions. This ensures that they only get
written when the transaction completes.
Indexedlog doesn't have support for pending writes, so mutation entries are not
visible to hook scripts.
Reviewed By: quark-zju
Differential Revision: D14566781
fbshipit-source-id: eb4c7bbd3878df82e8e7096a69509525f9fb93c0
Summary:
Remove the last parts of the mutationcache, relying directly on the indexedlog
indexes.
The set of obsolete commits is now computed on the fly - we will cache this
later.
Reviewed By: quark-zju
Differential Revision: D14566780
fbshipit-source-id: 770d0d9d507bed982e88518dcf63a4c2d6b46a69
Summary:
When visibility changes, the cache of filtered revs is no longer valid and must
be cleared.
Reviewed By: quark-zju
Differential Revision: D14566777
fbshipit-source-id: 681a73c126f3258baa7f61157aba557ea8aceeca
Summary:
To prepare for transaction support, we need to move the store out of the cache
object and into its own property on the localrepo.
Reviewed By: quark-zju
Differential Revision: D14566776
fbshipit-source-id: 8fff03a86953fb60ed06dbbdcdd0bffc379bd047
Summary:
Computing mutation entries for all local commits is expensive when there are
lots of local draft commits. Ideally we would have an indexed changelog that
would make these lookups fast, but until we have that, put entries in the
mutation store for these commits to take advantage of the fast lookup there.
Reviewed By: quark-zju
Differential Revision: D14566782
fbshipit-source-id: cc3a05715337a510a65d8ff436c59d16d0f0447e
Summary:
The computations for instabilities revsets (`orphan`, `contentdivergent` and
`phasedivergent`) are complex, and aren't really necessary for the way we use
mutation information.
The `orphan` revset can be implemented as an alias for `obsolete():: -
obsolete()`. The other conditions can be detected on a case-by-case basis
later if we need them.
Reviewed By: quark-zju
Differential Revision: D14566784
fbshipit-source-id: 60b8443ad4c0c82d8250d8e9a10e73fae770daa8
Summary:
To support future alternate hashing schemes, predecessor identities should be
prefixed with the hashing scheme in use.
Currently there is only one hashing scheme: `hg`, which is the Mercurial
hashing scheme.
Reviewed By: quark-zju
Differential Revision: D14566778
fbshipit-source-id: baaaf2f078886a1cc7ac20d12923a63b4da09db6
Summary:
The upstream has removed it in https://phab.mercurial-scm.org/D2393. Do the
same.
The deleted C++ code seems to leak `Py_False` if `usemanifestv2` is not set.
Reviewed By: singhsrb
Differential Revision: D14611525
fbshipit-source-id: d828526c31aaa861d100a846bba79d1f5898e245
Summary:
A user hit cryptic error message:
File "revset.py", line 423, in ancestor
ancestors = repo.changelog.index.ancestors
AttributeError: 'list' object has no attribute 'ancestors'
That was because they have run `rm -rf` and the `.hg` directory was mostly
removed. `hg` falls back to revlog v0, and the code in question obviously
dislikes revlog v0.
Change the error message to just say repo corruption.
Since there are code depending on revlog v1, and we don't use v0 or v2. Remove
both of them.
Reviewed By: singhsrb
Differential Revision: D14610769
fbshipit-source-id: 9015b74989d8fa4ad50bc36ec48bca26a1faa361
Summary:
Callers may call `remotenames.activepath` with the remote as a string URL. For
remotenames we must normalize these paths as we do for remotes that are passed
in as objects, for the same reason as in D14051479.
Differential Revision: D14621090
fbshipit-source-id: 8a02e9554039ad3daf3093001889cf1746ce395a
Summary:
The extdiff extension uses archive on the workingctx to make snapshots
of certain files. Prefetching fails on this because it sees placeholder file
nodes. Let's just not prefetch in this case at all.
Differential Revision: D14609691
fbshipit-source-id: 2d06bf015a8b10afa16b3b0bcb266afb102d63b9
Summary:
Now that we have perftracing infra, let's trace a bunch of likely
problem spots.
Reviewed By: sfilipco
Differential Revision: D14426367
fbshipit-source-id: 354a241aa9ac5d75d34062a9838d581b4f46746f
Summary:
Most of the repacks are background repacks, and most of the complaints are
coming from laptops users. Thanks to the rust repack, most of time during
repack is now spent in repacking loosefiles. While repacking them is expensive,
testing whether data is in a loosefile and obtaining it is actually pretty
fast. Packfiles have the opposite issue, repacking them is fast, but finding
data in them may be expensive if a lot of them are present.
Based on this, it makes sense to repack packfiles more frequently than
loosefiles. At first, the newly added option will be used to turn-off loosefile
repack for laptop users. A less frequent loosefile repack will be implemented
in a later patch.
Reviewed By: DurhamG
Differential Revision: D14586986
fbshipit-source-id: 5bc5c839cf8d2d78bcc4ffa016bbe3cf1b2ef3f7
Summary:
See D14606986 for context. `repo.close` should be protected by not crashing on
stdout or stderr write errors (SIGPIPE or EPIPE). Move `deferred` to `atexit`
handler to get the protection.
Test changes are caused by ordering changes.
Differential Revision: D14607407
fbshipit-source-id: 5a42aefcec395f48b8ecb67426429ef2e41f5666
Summary:
This is the first diff of an alternative fix of D14528603. See also D14603974
for context.
We'd like atexit handlers to not crash with EPIPE or SIGPIPE when writing to
stderr or stdout. Therefore disable SIGPIPE signal handler and patch ui methods
to do so.
Differential Revision: D14607159
fbshipit-source-id: 274c5174813d402a7e0b8b5be7c8fcb0524fcdb3
Summary:
akushner and I both hit an issue where `hg log` hangs for a long time after
exiting the pager by pressing `q`. That was caused by D14528603. Revert it to
fix it.
The root cause is apparently a bug in Python 2 (or C stdlib?):
# a.py
import signal, os
signal.signal(signal.SIGPIPE, signal.SIG_IGN)
out = os.fdopen(1, 'wb', 1)
while True:
out.write(b"a\n")
# shell
python2 a.py | head -n 1
# hangs
Python 3 does not have the issue.
Reviewed By: singhsrb
Differential Revision: D14603974
fbshipit-source-id: 3b4d716133853632e56dfeb59b1858f926a2c146
Summary:
Now that we're recording perftraces, let's log their results to the
blackbox if the command exceeds a configurable threshold.
Reviewed By: sfilipco
Differential Revision: D14426366
fbshipit-source-id: 24f261aaa2b089dbc959d709e51de1c0359d976d
Summary:
Adds tracing for the command dispatch. All other traces will be a
child of this root level trace.
Reviewed By: sfilipco
Differential Revision: D14426369
fbshipit-source-id: 2b996e027a2d6b163162bdceba571a3ee2186f6b
Summary:
Adds the initial perftracing infra. It's just a collection of functions
for starting and ending tracing and associating data with the traces.
Reviewed By: quark-zju
Differential Revision: D14426368
fbshipit-source-id: 72a2fad901ff52877f3958db2b924d95f45eeac7
Summary:
```
building 'indexes' library extension
warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
package: /data/users/sfilip/fbsource/fbcode/scm/hg/edenscm/hgext/extlib/indexes/Cargo.toml
workspace: /data/users/sfilip/fbsource/fbcode/scm/hg/edenscm/hgext/extlib/Cargo.toml
warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
package: /data/users/sfilip/fbsource/fbcode/scm/hg/edenscm/hgext/extlib/pyrevisionstore/Cargo.toml
workspace: /data/users/sfilip/fbsource/fbcode/scm/hg/edenscm/hgext/extlib/Cargo.toml
```
`profile` settings are ignored for non root packages. I introduced this issue
when I added a workspace for extlib: D14543989.
Reviewed By: kulshrax
Differential Revision: D14606019
fbshipit-source-id: 7ec4743d0913e443c378ae83f392817f6e6d3aab
Summary:
Closing a repo is required to commit the pending transactions, or to run a
background repack when packfiles were downloaded. Usually, the close method is
called in dispatch after a command is run and a valid repository was passed via
-R. The debugedenimporthelper however may open the repository manually, and
since it doesn't close it, we may end up with a large number of packfiles.
Let's manually close it to avoid this issue.
Reviewed By: simpkins
Differential Revision: D14531514
fbshipit-source-id: 9b2d1648775e4a2c9eea4c232bf0ae4fee43297d
Summary:
If the cloud sync or pushbackup commands start in background themselves, it will trigger a new background command to back up to the secondary afterwards.
Note that for cloud sync, for the secondary store we trigger pushbackup command, not cloud sync. If should be fine. We don't backup duplicates as we always check with the server first what is already backup. To call pushback looks simpler as it doesn't involve all the complexity of cloud sync, and basically we just need to backup to the secondary.
Reviewed By: markbt
Differential Revision: D14578770
fbshipit-source-id: f81a50fd76e64f2d77d8d7004b201fcfe8a090d2
Summary:
Change debugignore to test directories properly (which, only affects sparse
repos in practise) and stop printing out the source ignore rule location, as we
no longer have that information in the gitignore world.
Reviewed By: markbt
Differential Revision: D14547049
fbshipit-source-id: e7c307b1e1f4a8dcd30d00bec1f3e7a84a14b47b
