Summary:
While the Rust code can read/write content out of an indexedlog, the Python
code cannot. For now, all the writes will be done in Rust, and the Python code
will only be able to read from it.
Reviewed By: quark-zju
Differential Revision: D14894330
fbshipit-source-id: 5c1698d31412bc93e93dabb93be106a2ef17d184
Summary:
While not a correctness issue, taking and releasing the repack lock multiple
times per repack makes `hg debugwaitonrepack` unreliable, which in turns makes
test flakey.
Reviewed By: quark-zju
Differential Revision: D14877896
fbshipit-source-id: 682b649f388d19fd51bcf8dd205ac96944039e86
Summary:
With explicit visibility tracking, some commits may be hidden but not obsolete.
Pullcreatemarkers should not include these hidden commits when it is considering
what to link to landed commits.
Differential Revision: D14891062
fbshipit-source-id: 542363c979aca85d6de2e2161c00d91f67f0add3
Summary:
Add support for explicit visibility tracking in commit cloud sync.
This means commit cloud reads the visibleheads and syncs these with the commit
cloud heads directly, removing the source of problems where obsmarkers disagree
on different hosts.
Commit cloud requires that the ordering of heads is maintained to get stable
ordering of new commits. Update the visibleheads tracking to maintain
ordering, rather than using sets.
Finally, the calculation of the replacement node was slightly off. This was
revealed in the new test case that is being added, so it is also fixed.
Differential Revision: D14876266
fbshipit-source-id: fe5b6bffd196d3bd74e7582e29484969495eac8e
Summary: The `absorb` command should also record mutation information when it modifies commits.
Reviewed By: DurhamG
Differential Revision: D14871232
fbshipit-source-id: 46bc95b7f5781f0b5f5e057a34c755fcfe653f7e
Summary:
With explicit visibility tracking, the hiddenoverride feature of the amend
extension is no long necessary. When upgrading to explicit visibility,
include the pinned visible commits, however after that point, hiddenoverride
should have no effect.
Users can permanently reveal obsoleted commits using `hg unhide`.
Reviewed By: DurhamG
Differential Revision: D14851269
fbshipit-source-id: b516fa97ed1545d39b51baaf9c574cb73a571645
Summary:
When run with `--hidden`, the calculation for `backupheads()` may include extra
commits (ones that are hidden but not obsolete). This can cause some of the old
nodes to apper with "Not backed up" annotations. They should be excluded, too.
Reviewed By: DurhamG
Differential Revision: D14851035
fbshipit-source-id: f176b50a7ac62dc493a5c4df1e0c72532c43b680
Summary:
The pullcreatemarkers extension creates fake markers for landed commits by comparing
the differential revision of draft commits with those of landed commits.
In the future, mutation records will handle lands correctly, however until that stage,
replicate the pullcreatemarkers behaviour.
Reviewed By: DurhamG
Differential Revision: D14834746
fbshipit-source-id: e196ee28645270ccab1775ea896f0e453740c03e
Summary: This will help us debugging of what's going on on the server side.
Reviewed By: StanislavGlebik
Differential Revision: D14876167
fbshipit-source-id: 17aaa6dca9d5b5d7bbf69be46b58e454fe9f3fc3
Summary:
Record the commit predecessors and the mutation operation in the commit loginfo
so that it can be logged to telemetry.
Reviewed By: quark-zju
Differential Revision: D14798032
fbshipit-source-id: 9c4ac1a4df3c91087c776d1f8e5fca94713b0390
Summary:
Most of the extern crate are now removed. The cpython one is kept around as it
heavily depends on internal macros that would need to be manually imported.
Reviewed By: quark-zju
Differential Revision: D14864995
fbshipit-source-id: 05405ac7310e4dca65daf230cc8f574da32ed4c9
Summary: Make the batch size of data and history requests independently configurable, since data responses are typically much larger than history responses (since the former contains actual file data whereas the latter is only metadata).
Differential Revision: D14859686
fbshipit-source-id: c87c31f3e6611a55ae712e7f0ed9bb392d31a579
Summary: Use the curl multi interface to fetch multiple batches of files or history entries concurrently.
Differential Revision: D14718547
fbshipit-source-id: c5a740c7e9106b719e825540f8182be31a72bae7
Summary:
The command is supposed to check if the given commits are backed up and if not to back them up.
This is needed for `jf submit` to make sure that all the commits submitted are backed up by hg.
Reviewed By: markbt
Differential Revision: D14685222
fbshipit-source-id: 224a438ec1c5c17af6988c511be4af4a3a988a79
Summary:
Currently we skip metadata when constructing data wirepacks.
We need copy metadata in the data packflies, because the client expects the base text to contain copy metadata since the delta base on the server also contains copy metadata.
It is also needed for future file nodes validation.
Differential Revision: D14851678
fbshipit-source-id: 1a3f79dc2565cdb864bee2400d331ae3a7c3751b
Summary:
The Dxxxx revset does two things in parallel:
* walks the changelog in the search of commit associated with Dxxx.
* makes graphql query to resolve Dxxxx to the hash associated with the last
submitted version.
This behaviour is clowny and confusing and should be changed long-term. This is
just a short term fix to prevent it from crashing in cases there's no local
commit associated with Dxxx (or it's very old) and the graphql query comes back
with empty result (because there are no versions for the diff).
Reviewed By: markbt
Differential Revision: D14851313
fbshipit-source-id: 14f7531aa4691007266291a38a016242de563de4
Summary:
When determining the contexts to compare trees against, consider all parents
of the missing nodes that aren't themselves missing.
In the case of:
```
C
/ \
D B
|
A
```
Where A and D are common and B and C are missing, the parents of the roots
are only A, which means D's tree will not be compared against and all of its
trees will be included.
Reviewed By: StanislavGlebik
Differential Revision: D14798681
fbshipit-source-id: c80add907338a3011daf54a2304ea19415b7f9fb
Summary:
The debugbundle implementation for mutation parts was not updated with
the recent changes made to mutation entries. Update it.
Reviewed By: quark-zju
Differential Revision: D14798682
fbshipit-source-id: eb1defa33e8a0eed807360af2c130a225af15030
Summary: Allow users to configure which HTTP client backend to use for the Eden API via the `edenapi.backend` config option. Valid options are `curl` and `hyper`, with `curl` being the default.
Reviewed By: quark-zju
Differential Revision: D14657871
fbshipit-source-id: 7a9972d2380fbbd5ed62d1accae764dc03ca4c29
Summary:
In most cases, we won't have more packfiles than what is held in the LRU cache,
and therefore, building a set is unecessary. Testing for the length is a O(1)
operation and is therefore a quick test to verify if more packs needs to be
iterated on.
Reviewed By: quark-zju
Differential Revision: D14802940
fbshipit-source-id: 8b39befb27368d474ab71f3eeac2340d8183e70e
Summary:
When many packfiles are present, and searching for missing data, we will
iterate on the packs twice. We can avoid the second iteration by testing if new
packfiles were detected.
Reviewed By: quark-zju
Differential Revision: D14802942
fbshipit-source-id: 624d9a86e65395af3141ea10d42bcfb8ee18db83
Summary:
Building a set is a O(n) operation, and in the cases where refresh is called
often, it starts showing up in profiles. Instead of rebuilding it everytime, we
can simply update it when adding/removing packs.
Reviewed By: quark-zju
Differential Revision: D14802945
fbshipit-source-id: f8967d09d2d7d0cc0d7400b047a587d536315002
Summary:
The repo.ui.configbool was showing in profiles when running `hg log -p`. Since
its value won't change for the duration of the operation, let's cache it.
Reviewed By: quark-zju
Differential Revision: D14802941
fbshipit-source-id: c1a2a764cb4d4a1c0d45d5118bb1e892c10798a6
Summary:
Add the checkout identifier to the loginfo for commit contexts that take data
from the working copy.
Reviewed By: quark-zju
Differential Revision: D14751601
fbshipit-source-id: 6ddcbf17b970ee24237edf501ae3d7fc8d320193
Summary: Extract the `makerandomidentifier` function from the `clienttelemetry` extension so that we can use it elsewhere.
Reviewed By: quark-zju
Differential Revision: D14725866
fbshipit-source-id: 0b3ed2e8c9698e892ad0f524f6dde7bf7337f85c
Summary:
One of the drawback of only using packfiles is the cost of the pathological
cases when one pack file is being downloaded per file. This can happen during a
`hg log -p`, or on Eden when running `cat` on lots of not materialized files.
In these cases, mercurial will become slower and slower as the command is
running.
This is due to mercurial having to search each individual packfile to find
whether some data exist locally, or it needs to be fetched from the network.
This search will iterate over each packfile linearly, and on a miss, a
directory scan will be performed to see if a new packfile is present.
Previous patches attempted to solve this by running a repack when the command
finishes, but in the case of `hg log -p`, or when Eden is requesting the
content of some file, the command could be running for a long time.
As a solution, let's repack while the command is running if we detect that too
many pack files are present.
Reviewed By: quark-zju
Differential Revision: D14740566
fbshipit-source-id: 95ba381bd99e5404e352f799e9053a6375abac0a
Summary:
Mercurial recently started to generate empty pack files (`0x01`). This will cause this check to fail:
diffusion/FBS/browse/master/fbcode/scm/hg/lib/cdatapack/cdatapack.c;2c4197d003ed906dd8eaf70fbb04aa53440ce681$314-319
This will subsequently result as a double-free error between these two:
**In `error_cleanup`**
diffusion/FBS/browse/master/fbcode/scm/hg/lib/cdatapack/cdatapack.c;2c4197d003ed906dd8eaf70fbb04aa53440ce681$387-389
**In `close_datapack`**
diffusion/FBS/browse/master/fbcode/scm/hg/lib/cdatapack/cdatapack.c;2c4197d003ed906dd8eaf70fbb04aa53440ce681$401
This diff will fix this bug.
Differential Revision: D14759374
fbshipit-source-id: 06f192513a935740c2142b5a2baac87a28903496
Summary:
The phabricator team is trying to eliminate `DifferentialRevision` in favor of `PhabricatorDiff`
See context here
https://our.intern.facebook.com/intern/wiki/Dev_Tools/Phabricator/GraphQLObjectRename/
I'm using aliases to reduce the number of changes needed (the object from the graphql query will have the same keys)
Reviewed By: quark-zju
Differential Revision: D14703867
fbshipit-source-id: 32204315dc154db7cbff68ee286579c0200f57b2
Summary:
Loosefiles are quite slow to repack, and this is significantly affecting
laptops as the repack can potentially run for hours when a large amount of
loosefiles are present.
One of the reason for that is that an incremental repack really isn't
incremental for loosefiles as all of them are repacked at the same time.
Instead let's repack only one of the top-level directory at a time.
Reviewed By: quark-zju
Differential Revision: D14690348
fbshipit-source-id: ffba49840302ae0d99e32db410647e83e213fe64
Summary:
Currently, if you lookup a globalrev which does not exist in the
repository, it leads to a linear scan of the entire changelog even while using
the globalrev based fast commit lookup. This commit speeds up the same by
relying on the indexing information.
Differential Revision: D14726937
fbshipit-source-id: 076ad8608aa656075dcb2c1287502ffe93bed851
Summary:
This is just pending feedback from D14694620 which makes the class
interface more cleaner.
Reviewed By: quark-zju
Differential Revision: D14718748
fbshipit-source-id: 032fc25d7e5922aac5f4195eb007bd009822cb86
Summary:
When the sql_repo_lock.py hook is run, the sql connection won't have been
established yet. Let's open a connection and clean it up afterwards.
Reviewed By: quark-zju
Differential Revision: D14708851
fbshipit-source-id: f20b6dacdcb12cee839163325164d2e75816100c
Summary:
The type for globalrev during lookup should be an integer instead of a
string after D14694623.
Reviewed By: quark-zju
Differential Revision: D14718749
fbshipit-source-id: eddc49ea0faa61df6ad435db2192fba4f61b0075
Summary:
Importing an extension should be side-effect free. Move the patching to uisetup
so they only get executed when the extension is enabled.
Reviewed By: kulshrax
Differential Revision: D14694622
fbshipit-source-id: c2802e0f4d852445dd7900f11d4ede6e58883d94
Summary:
Change the index key from decimal to binary to make the index more compact and
efficient. Move the binary to the beginning to avoid common prefixes and speed
up lookups.
Reviewed By: singhsrb
Differential Revision: D14694623
fbshipit-source-id: e908d287c1d3559639dbafa39a465f89b8f6f14e
Summary: This makes the progress visible.
Reviewed By: singhsrb
Differential Revision: D14694621
fbshipit-source-id: bb80546cb513c423471ba004bf00db7431046bcf
Summary:
Use revision numbers to make sure the index contains all commits.
Also include svnrev in the same index. This has multiple benefits:
- Handles svnrev mapping, too. Easier to drop hgsubversion.
- Handles the special commit without globalrev or svnrev gracefully.
It takes about 3 minutes to build the index for www.
I removed code about phases, or heads. The code runs on hg servers
and hg servers (hgsql) have hard dependency on rev numbers.
If we get a chance to migrate away from client-side local changelog,
we'll have Mononoke equivalent commit mapping by that time (or, push
commit mapping work to other teams).
Reviewed By: singhsrb
Differential Revision: D14694620
fbshipit-source-id: 9b74782f4c37b6620ab7168f7c7199d0cc5bc396
Summary:
This is a requirement to enable Phabricator migration to a globalrev
based repository. The release engineering team has also expressed interest in a
faster way to lookup the commits using globalrev.
I am just using indexed log for now to enable faster lookup. We might not need
to backfill the indexed log since we mostly care for the newer commits. But if
we do need to, we can do it later.
Reviewed By: quark-zju
Differential Revision: D14687707
fbshipit-source-id: fe02e9a72b9edb57cfd49a294642014c7204c058
Summary:
This is required for the fast commit lookup based on globalrev in
D14687707. This code is heavily inspired from DurhamG's implementation for the
git-hg mapping in the `hggit` extension.
quark-zju has good suggestions for improving this to include the svnrev as well
which can eliminate the requirement for a svnrev-hg mapping in the future. We
can certainly do those in a later iteration.
Reviewed By: quark-zju
Differential Revision: D14674111
fbshipit-source-id: d09d9e65ac0abcafd478c873086898f787398867
Summary: This is just for convenience and making the code cleaner.
Reviewed By: quark-zju
Differential Revision: D14689470
fbshipit-source-id: 911f49e24a3bb504098537169e9096b080a86535
Summary:
Allow running `hg push --to scratch/bookmarkname -r X` where X is a public
commit.
Note that updating a scratch bookmark that is on a public commit will
require a force push.
Reviewed By: quark-zju
Differential Revision: D14527730
fbshipit-source-id: dbc234b43d74a36b7403bea19e1e917ad69cdbbf
Summary: Previously the debug commands for the Eden API did not check whether HTTP data fetching was enabled before trying to access the client singleton, which would result in a crash if HTTP was disabled. This diff adds explicit checks so that the commands will abort instead of crashing.
Reviewed By: quark-zju
Differential Revision: D14692893
fbshipit-source-id: d36e241d8460dadb555a15c92aca9334ba00f34c
Summary:
With `remotefilelog.fetchpacks`, some pathological cases would end up creating
tons of packfiles and we would thus spend a significant amount of time in
refresh, scanning for new packfiles. Jeremy noted that for every refresh,
stat(2) would be ran on every file, even though only a handful of packfile
would be added.
Differential Revision: D14674093
fbshipit-source-id: df198d208e3f4e1d667e7bdd069a793984683282
Summary:
Now that repacks runs more often, it's easier to trigger a race between repack
deleting packfiles, and another mercurial process listing the packfiles and
trying to open them. If the packfiles are deleted after the directory listing,
all the packfiles will fail to be opened and were mis-reported as corrupted.
Reviewed By: quark-zju
Differential Revision: D14648308
fbshipit-source-id: c3b852f669e28db6f622bde217f339533e094223
Summary:
Add basic support for `undo` by updating visibility according to what the undo
extension would normally do with obsmarkers.
A future enhancement would be to use the `draftheads` directly as the new
visibleheads, however:
* `undo` currently uses `heads(draft())` which doesn't take into account
secret commits.
* In order to support the `--branch` flag, we would need to make significant
changes. The flag is marked as ADVANCED and isn't used very much, so we may
consider removing it.
Reviewed By: quark-zju
Differential Revision: D14603196
fbshipit-source-id: 5b8c91b4d8fa2b28cf9ad5726d88a3735aca50ad
Summary:
Computing mutation entries for all local commits is expensive when there are
lots of local draft commits. Ideally we would have an indexed changelog that
would make these lookups fast, but until we have that, put entries in the
mutation store for these commits to take advantage of the fast lookup there.
Reviewed By: quark-zju
Differential Revision: D14566782
fbshipit-source-id: cc3a05715337a510a65d8ff436c59d16d0f0447e
Summary:
The upstream has removed it in https://phab.mercurial-scm.org/D2393. Do the
same.
The deleted C++ code seems to leak `Py_False` if `usemanifestv2` is not set.
Reviewed By: singhsrb
Differential Revision: D14611525
fbshipit-source-id: d828526c31aaa861d100a846bba79d1f5898e245
Summary:
Callers may call `remotenames.activepath` with the remote as a string URL. For
remotenames we must normalize these paths as we do for remotes that are passed
in as objects, for the same reason as in D14051479.
Differential Revision: D14621090
fbshipit-source-id: 8a02e9554039ad3daf3093001889cf1746ce395a
