Commit Graph

26 Commits

Author SHA1 Message Date
Jun Wu
cd04a56178 repo: avoid metalog when migrating files from sharedvfs to storevfs
Summary: Otherwise it can trigger metalog ProgrammingErrors.

Reviewed By: markbt

Differential Revision: D19624779

fbshipit-source-id: ba56a9aefa28d56a38ea8d966115647975613c91
2020-01-30 07:25:31 -08:00
Durham Goode
38756127f2 hgevents: disable events during status
Summary:
There are times when status updates the dirstate to mark files as
clean or to resolve lookup files. In doing so it takes the wlock, which triggers
a state-enter event which causes external watchers to then run hg status, which
can then trigger another wlock event from status. This causes an infinite loop.

It's not clear why status is taking the wlock every time. It should be a
one-time thing. So more investigation is needed. For now though, we can prevent
this from happening by disabling hgevent signals while status has the wlock.

Reviewed By: quark-zju

Differential Revision: D19602370

fbshipit-source-id: 609316de1441d40be4f81236b8d00ce7dc30a130
2020-01-29 10:30:22 -08:00
Jun Wu
751d53d4e6 py3: localrepo: fix some encoding issues
Reviewed By: farnz

Differential Revision: D19615825

fbshipit-source-id: 1801652ed93ac9218a66af7f23f5bce9eab3ab10
2020-01-29 07:45:17 -08:00
Genevieve Helsel
ce2bf270b3 fix_map
Summary: Python 3 map call doesn't return list, so wrap in list.

Reviewed By: mitrandir77, xavierd

Differential Revision: D19588352

fbshipit-source-id: 8556f4783f9bedb8a74ec0e99ff226ff96f387a9
2020-01-28 18:24:56 -08:00
Mark Thomas
82715fd2ea py3: iter{keys,values,items} -> pycompat.iter{keys,values,items}
Reviewed By: quark-zju

Differential Revision: D19608323

fbshipit-source-id: dd186ef16d6422a56af41fcaa850d9838ae9a240
2020-01-28 16:27:28 -08:00
Jun Wu
35250bd8fd py3: miscellaneous fixes
Summary: Some miscellaneous fixes, ex. `map(x)` -> `list(map(x))`.

Reviewed By: farnz

Differential Revision: D19581171

fbshipit-source-id: afbb3b8b798f7878fc3206586497ea771116066b
2020-01-28 10:29:14 -08:00
Mark Thomas
39f8b3109f visibility: add visibleheads to journaled files
Summary:
The visibleheads file should be journaled like the other state files, so that
it is rolled back if a transaction aborts.

Reviewed By: quark-zju

Differential Revision: D19502226

fbshipit-source-id: c2b8d926ce97211411ca01f05e83e62cb77625b9
2020-01-24 14:47:37 -08:00
Jun Wu
4a98b00205 changelog: still migrate non-public commits to zstore
Summary:
This makes it safer to do revlog-fallback -> server-fallback switch in the
future, because the server might not know all the draft commits.

Reviewed By: DurhamG

Differential Revision: D19416287

fbshipit-source-id: 8d69c1c5465b50710110370d84d5fb7c8ba6a6c4
2020-01-17 04:01:05 -08:00
Mark Thomas
eef0c650e7 mutation: make the mutationstore the source of truth for mutation entries
Summary:
Previously the mutation commit extras were the source of truth for mutation
information, and the mutation store served as a kind of cache.  This turned out
to be less useful than expected, as oftentimes commits are missing, and the
store is better indexed, so in practice using the store as the source of truth
is better.

This change makes the mutationstore the (sole) source of truth for mutation
data.  The extras are kept, but they are now only useful as human-readable
debug information, and to ensure the commit hash is unique.

Collecting the mutation information during commit creation is now done through
a new `mutinfo` object.  This is a dict with the same keys as the mutation
extras, for simplicity, but it is now passed through the `committablectx` and
used to generate the mutation store entry directly.

The `mutation.enabled` config option is now used to control all aspects of
enabling mutation.

The `mutation.record` config option is now only used to indicate whether the
mutation extras should also added to the commit.  Generally this should be set
to `true`, however the option is retained so that mutation extras can be
stripped by running `hg amend --config mutation.record=false`, which no longer
has the side-effect of not recording mutation information to the store.

The "remote commit" mutation record origin is now obsolete, and won't be
generated anymore.

Pushrebase now relies on the obsmarker information coming back from the server
in order to correctly generate mutation information.  We will need to change
this so that the server returns mutation records before we can fully deprecate
obsmarkers.

Reviewed By: DurhamG

Differential Revision: D19410650

fbshipit-source-id: 8d7094e4bfd8d8e97916898d899a8debd339485f
2020-01-16 01:03:22 -08:00
Josh Rosenbaum
6bf2ce6f9b log common directory path of all files in the changeset
Summary: Begin logging the common directory path for the commit's updated files.

Reviewed By: DurhamG

Differential Revision: D19399717

fbshipit-source-id: d47994f573eac6e7cd1596b0b102cb9577e1eea1
2020-01-15 19:06:17 -08:00
Jun Wu
2ade5663b1 mq: remove references about mq
Summary: `mq` was removed. Remove remaining references to it.

Reviewed By: singhsrb

Differential Revision: D19382027

fbshipit-source-id: d194556b63dd6b9a44b949bd24b7e1a7356d5c04
2020-01-13 17:39:43 -08:00
Durham Goode
e959466b51 automigrate: don't call automigrate if lock is held
Summary:
We don't want automigration to block normal execution of the command,
so let's take the lock with a short timeout, then skip automigration if we don't
get it.

This should solve an issue where debugedenimporthelpers were hanging because
they tried to take a lock that a hg process that was accessing eden held.

Reviewed By: quark-zju

Differential Revision: D19375544

fbshipit-source-id: 6c107aac87b5b839cc2d347a3b40ce67da627b71
2020-01-13 13:52:18 -08:00
Jun Wu
ffc85f7222 changelog: add a way to migrate up and down with zstore-commit-data format
Summary:
This makes it possible to use zstore instead of 00changelog.d to store commit
data. Right now it double writes to zstore and 00changelog.d if turned on.
In the future we can switch to only writing to zstore if 00changelog.i
alternative is available.

Some related configs were added for fallback strategy: revlog or server.
The revlog fallback allows us to do fast migration. The server fallback
(not implemented in this diff) allows us to stop depending on revlog.d.

Reviewed By: DurhamG

Differential Revision: D18737260

fbshipit-source-id: 3c9605f0babd8a215ee74bdf1275cc4e9dbd766c
2020-01-10 19:01:43 -08:00
Josh Rosenbaum
6239a21832 Add "Diff Number" column to "perfpipe_hg_commit_info"
Summary: I would like to link the Phabricator Diff Number to specific mercurial revisions.  After talking to Durham, the least invasive and best method for accomplishing this is to log the Phabricator Diff Number at runtime. Hence, whenever someone modifies the current commit, log the Phabricator Diff Number (when possible). The resulting table will theoretically have a map from "revision" -> "Phabricator diff number".

Reviewed By: DurhamG

Differential Revision: D19334379

fbshipit-source-id: eb04a8af4dbeb9a7e130a268fce8a6c5d26ebfa9
2020-01-09 17:15:13 -08:00
Durham Goode
d8663cb534 treestate: backout making it a hard requirement
Summary:
It turns out some old repos are still using the legacy treedirstate
format, which broke when I made this a hard requirement. Let's roll this back
for now.

Reviewed By: singhsrb

Differential Revision: D19313773

fbshipit-source-id: ded78be033cb04323a2e1d237059947dee509df9
2020-01-08 11:33:28 -08:00
Durham Goode
9778811573 tags: remove tags feature
Summary:
We no longer user tags in any of our repositories, so let's delete all
the code and remove some tech debt.

Previously drawdag was our last remaining use of tags, but in a previous diff I
switched that to use bookmarks.

Reviewed By: quark-zju

Differential Revision: D19042226

fbshipit-source-id: d7871c71c1026599aa70fdd806418347ea214a8a
2020-01-07 12:31:10 -08:00
Durham Goode
898149fa37 treestate: make treestate a hard requirement
Summary:
We want to delete all the non-treestate dirstate implementations. Let's
start throwing an exception if treestate is not enabled. We temporarily have a
bypass in case we break an important usecase in the process.

This also sets the standard new repo to be created in treestate mode, but adding
treestate to newreporequirements.

Reviewed By: quark-zju

Differential Revision: D19204621

fbshipit-source-id: 61c0c69ce6e559615ef570d2343bea78c1b5d5ee
2020-01-06 21:37:05 -08:00
Durham Goode
218c83d574 tags: remove tip tag
Summary:
In a future diff we'll be removing tags. The most prevalent tag is
'tip', which shows up in a ton of test output. Let's drop that tag first, so we
can safely update the tests before we drop tags entirely.

Reviewed By: xavierd

Differential Revision: D18995058

fbshipit-source-id: 8c63710cd4ed567ea24e32724b8660f9006a61f1
2019-12-20 16:14:19 -08:00
Xavier Deguillard
81365e9e9f localrepo: use repo.commitpending in repo.commitnotransaction
Summary:
There is no need to duplicate the code.

The test change are due to commitpending being called 3 times, vs 1 before. The
total amount of times {fileslog,manifestlog}.commitpending is called is unchanged.

Reviewed By: quark-zju

Differential Revision: D17841316

fbshipit-source-id: be33a126f913af8824a6d5ade17351707d39ed2f
2019-12-12 14:15:05 -08:00
Xavier Deguillard
ff36d65b5d typing: enable pyre
Summary:
The Mercurial codebase contains over 500 errors, let's ignore them for now, we
can go back to them later to fix them.

Besides the manual change to .pyre_configuration.local, the changes were
generated with:
  pyre --output=json check | pyre-upgrade fixme

Reviewed By: singhsrb

Differential Revision: D18803908

fbshipit-source-id: 724db7bd864c0de47a97ef2092bdee9f2cda531f
2019-12-04 10:55:00 -08:00
Jun Wu
c0bc1fb8be bookmarks: move 'bookmarks' state to storevfs
Summary:
Similar to D17199834, by moving 'bookmarks' from localvfs and sharedvfs to
storevfs, we can make it part of metalog and get better transaction support.

Reviewed By: xavierd

Differential Revision: D18524104

fbshipit-source-id: ae148c1f02fc83b5c2d73102ecab39ff223ea5df
2019-11-27 11:37:43 -08:00
Jun Wu
4fe519d97f repo: always share bookmarks
Summary:
Right now `bookmarks` is stored at both "local" vfs and the "shared" vfs.
In an upcoming change I'd like to move bookmarks to the "metalog" to get
better transaction. Therefore make it always shared.

The shared bookmark feature is turned on by default. Assuming nobody manually
turns it off, production hosts all have shared bookmark turned on.

Reviewed By: simpkins

Differential Revision: D18524103

fbshipit-source-id: 3160146f357dd01f2d78f5d2f4a14838f8f12a94
2019-11-27 11:37:42 -08:00
Jun Wu
9c8156804e metalog: raise ProgrammingError on misuse
Summary:
The only valid place that metalog can be changed is inside a transaction, since
it's transaction.{close,abort} that writes or discards metalog changes.

In other words, metalog should not be changed during A, B, or C:

  |<- A ->|<----------- repo lock --------->|
          |<- B ->|<- transaction ->|<- C ->|

Add detection for them.

Reviewed By: xavierd

Differential Revision: D18538143

fbshipit-source-id: 036286ed32a897fe3ce0a91c1e3c848cc6167b1d
2019-11-27 00:24:09 -08:00
Jun Wu
05b53a4d7b metalog: discard in-memory metalog at lock/transaction boundary
Summary:
Before this diff, metalog has a same lifetime as repo.svfs and never gets
reloaded or dropped. That is problematic in case external processes also
make changes, ex. running `hg amend` in `histedit exec`.

    pid 1> histedit action foo (a single transaction, metalog loaded here)
    pid 1> histedit action exec ... (a single transaction)
    pid 2> hg amend (changes metalog)
    pid 1> histedit action bar (cannot perform this action because metalog has
           conflicts!)

This is why test-mutation.t didn't work with metalog.

Fix it by discarding metalog state at transaction/lock boundary (enter/exit).

The next two diffs add checks so we wouldn't discard uncommitted data silently.

Reviewed By: xavierd

Differential Revision: D18519995

fbshipit-source-id: ceb030362d66ad4be142e81accb82a4afa67f305
2019-11-27 00:24:08 -08:00
Jun Wu
6b8acfe260 hgsql: add rate limiting for read commands
Summary:
Similar to D2756217. But skip sync directly if the repo has been synced
recently.

This will help reducing CPU load on the DB tier.

Reviewed By: ikostia

Differential Revision: D18621013

fbshipit-source-id: 83370669ea61eb693d06495aaa13664648b97aca
2019-11-21 07:45:31 -08:00
Adam Simpkins
ab3a7cb21f Move fb-mercurial sources into an eden/scm subdirectory.
Summary:
In preparation for merging fb-mercurial sources to the Eden repository,
move everything from the top-level directory into an `eden/scm`
subdirectory.
2019-11-13 16:04:48 -08:00