Commit Graph

1882 Commits

Author SHA1 Message Date
Jun Wu
c72cd2333f metalog: remove conditional metalog logic
Summary:
Now metalog can no longer be `None`. Let's just remove logic handling its
`None` case.

This changes the commitcloud-sync-race test because the metalog itself has
internal locking and changes are atomic.

Reviewed By: DurhamG

Differential Revision: D28595292

fbshipit-source-id: bd9851f5f3bb25f28f15d673f608af2863953c46
2021-05-25 19:43:45 -07:00
Jun Wu
1f8d8cfbf0 store: always enable store and fncache
Summary:
fncache and store have been default on for years. Enable them unconditionally.
This also makes sure that metalog is always available.

Practically, the only place that does not use fncache is hgsql server repos and
they are irrelevant now.

Reviewed By: DurhamG

Differential Revision: D28595289

fbshipit-source-id: 32b9906c179518acdb17a206b54f98a3dc994921
2021-05-25 19:43:45 -07:00
Jun Wu
2aa432e134 clone: add a config to clone with lazy changelog
Summary: This will be used by fbclone to ship lazy commit hash backend.

Reviewed By: DurhamG

Differential Revision: D28554445

fbshipit-source-id: a263ae7683124b3b86f4025b02c7de20dcb9813e
2021-05-24 14:51:58 -07:00
Jun Wu
5edc1f6433 metalog: expose compact as debugcompactmetalog command
Summary: This makes it possible to use non-debugshell to compact the metalog.

Reviewed By: DurhamG

Differential Revision: D28550902

fbshipit-source-id: 789830ba35243d248397e6a52ee343584c1e01a9
2021-05-24 11:44:55 -07:00
Jun Wu
bd1fb8e337 zipimport: workaroud pyc invalidation due to timezone settings
Summary:
The zipimport logic requires the pyc mtime to match its source. However, the
Windows system time zone can invalidate it and cause slow startups.

Workaround it by making the zipimport mtime function return a fallback value so
the mtime check is then bypassed.

    # zipimport.py, _unmarshal_code
    source_mtime, source_size = \
        _get_mtime_and_size_of_source(self, fullpath)

    if source_mtime:  # if source_mtime is false, then the check is bypassed.
        # We don't use _bootstrap_external._validate_timestamp_pyc
        # to allow for a more lenient timestamp check.
        if (not _eq_mtime(_unpack_uint32(data[8:12]), source_mtime) or
                _unpack_uint32(data[12:16]) != source_size):
            _bootstrap._verbose_message(
                f'bytecode is stale for {fullname!r}')
            return None

Change my Windows time zone from GMT-7 to GMT-4. Set PYTHONVERBOSE and
PYTHONDEBUG to 1. Ran `hg init -h` and check its stderr. It prints:

  # bytecode is stale for 'edenscm.traceimport'

and alike before this change, and no longer after replacing the `__init__.py`
in the zip with the new version.

Reviewed By: DurhamG

Differential Revision: D28622287

fbshipit-source-id: bb3e8e378ea168e4f83f4b6aa9713103b2c90ef8
2021-05-24 09:36:37 -07:00
Johan Schuijt-Li
112008bf7a mononokepeer: fix error message
Summary:
Would previously fail with:
  AttributeError: 'timeout' object has no attribute 'message'

Reviewed By: mzr

Differential Revision: D28636534

fbshipit-source-id: 44df954b659cfbfbe0b468798e7eb5142478be67
2021-05-24 05:34:10 -07:00
Liubov Dmitrieva
9f3609ec51 implement omision of old public bookmarks
Summary:
don't apply an old public bookmark if the commit is older than max_sync_age

there is a complicated logic because we need to make sure if we later run it with different commitcloud.max_sync_age value or hg cloud sync --full the bookmarks will be appear back.

So, the changes required in both:

* checkomission
* _mergebookmarks

but both cases covered in the tests

also, if you run with max_sync_age=1000 and later max_sync_age=0, the bookmarks will not disappear, which is expected.

Reviewed By: markbt

Differential Revision: D28572875

fbshipit-source-id: 317e897a2b81c3371dbea7eb39b8925570c1d40a
2021-05-21 06:31:04 -07:00
Liubov Dmitrieva
6a4814e664 reduce output of omitted bookmarks
Summary:
this output is not noisy for big workspaces

if a head is omitted, don't warn about the bookmark because it is expected

Reviewed By: markbt

Differential Revision: D28568919

fbshipit-source-id: eb19e1d155f65de411c1dd41a8be6d83ca71c264
2021-05-21 06:31:04 -07:00
Liubov Dmitrieva
bf0dab696c reduce output for listing omitted heads and bookmarks
Summary:
The output is a bit too noisy for large workspaces.

We could skip older commits, we know the list coming ordered from commit cloud service.

All hashes are anyway available via `hg cloud sl`.

Also I fixed several look-ups in the list. Omittedheads are heavily used to check if something is present there.

Reviewed By: markbt

Differential Revision: D28568421

fbshipit-source-id: bcf62522798fed92df7ca546c73aa14da95f1567
2021-05-21 06:31:04 -07:00
Carolyn Busch
4e03bd4021 pull: use bookmark http api
Summary: Add config pull.httpbookmarks to use the edenapi http protocol to fetch bookmarks in the central local repo pull method. This impacts the pull command, as well as other commands that pull bookmarks.

Reviewed By: quark-zju

Differential Revision: D27479112

fbshipit-source-id: 2b9821f458ec0af2579143fb2c2ed7d3ff41878a
2021-05-20 17:28:51 -07:00
Johan Schuijt-Li
daf7eb2138 add decompression support for mononoke
Summary:
Support decompression for mononoke connections. When we request it, Mononoke
can support compression our stream, saving bandwith on low throughput
connections.

Reviewed By: StanislavGlebik

Differential Revision: D28535058

fbshipit-source-id: 7594f72978093a474efd168bb87b41c415310d6c
2021-05-20 10:22:56 -07:00
Jun Wu
6ce450753a commitcloud: add FakeCtx.mutable
Summary: It can be used by `cloud ssl` template after D28000088 (b506eeea0c).

Reviewed By: liubov-dmitrieva

Differential Revision: D28561180

fbshipit-source-id: fb4bf3de85f7c320c13a2a53c6a103e85ebb5425
2021-05-20 09:44:41 -07:00
Jun Wu
7001f0de8a debugchangelog: add ways to skip migrating from specified format
Summary:
This allows us to do staged rollout where some users are using "lazy" backend
and they won't be migrating down to "doublewrite" backend.

Reviewed By: liubov-dmitrieva

Differential Revision: D28554381

fbshipit-source-id: ebe2e25c96fd3b086a451c3909643d19c64a186c
2021-05-20 09:42:12 -07:00
Jun Wu
5f9559ad99 debugchangelog: migrate from lazy to lazy is not an error
Summary: Migrating from the lazy backend to lazy backend should be a no-op.

Reviewed By: liubov-dmitrieva

Differential Revision: D28554382

fbshipit-source-id: 71c06584f6f7a89096ce4a94843c88cbea542475
2021-05-20 09:42:12 -07:00
Liubov Dmitrieva
12c4a1bbb1 add a config option to show all bookmarks in output of hg cloud sl
Summary:
add a config option to show all bookmarks in output of `hg cloud sl`

by default local bookmarks pointing to public commits are not returned unless it's a public root of some draft stack

Reviewed By: markbt

Differential Revision: D28537657

fbshipit-source-id: 0287c18b1b6c79b271f8a67f604024086a37ffcf
2021-05-19 10:01:19 -07:00
Liubov Dmitrieva
9f9bcd2c70 check commit cloud workspace for removing irelevant branches
Summary: If you have checked out a shared workspace or other user workspace, this part of hg doctor could hide incorrectly, so, should be skipped.

Reviewed By: markbt

Differential Revision: D28505928

fbshipit-source-id: 65e1b3978a916fad2a33bb4f81ff1b75cd657567
2021-05-19 07:03:21 -07:00
Carolyn Busch
411ed7adf5 infinitepush: use bookmark http api
Summary: Fetch bookmarks via the http edenapi protocol in the bookmark command with the --list-remote option when all bookmark patterns are full bookmark names (not prefixes).

Reviewed By: kulshrax

Differential Revision: D27331526

fbshipit-source-id: 4f4eda255c551c9b55c6966569755f493335b458
2021-05-18 11:10:33 -07:00
Mark Juggurnauth-Thomas
5dafb7333f commitcloud: support --workspace-version in hg cloud sl --history
Summary:
The --workspace-version option is currently ignored by interactive history.
Allow it to be used to specify the initial version.  This makes jumping back to
a much older version easier.

Reviewed By: liubov-dmitrieva

Differential Revision: D28478194

fbshipit-source-id: f4f121d919e89c298677256f227f2e96d63ef644
2021-05-18 09:16:15 -07:00
Liubov Dmitrieva
add020c9f3 increase commit cloud history limit for 12 weeks by default
Summary: increase commit cloud history limit for 12 weeks by default

Reviewed By: markbt

Differential Revision: D28476878

fbshipit-source-id: d66d2bd672232932b698fece143eeac82d426886
2021-05-18 02:11:53 -07:00
Liubov Dmitrieva
4fb70336da add an option to show remote bookmarks in commit cloud smartlogs
Summary: if this option is enabled, the server will be asked to add them

Reviewed By: markbt

Differential Revision: D28412810

fbshipit-source-id: d1531ecf97615cdb5e32d72c8c31598e6a406956
2021-05-17 08:39:31 -07:00
Durham Goode
d88b999faf Allow conflict editor prompts during arc pull
Summary:
Mercurial has gotten stricter about respecting interactive vs
non-interactive commands lately, and now is failing to automatically open the
editor for conflicts during arc pull. Let's force Mercurial to treat the
invocation as an interactive one.

Reviewed By: skotchvail

Differential Revision: D28358999

fbshipit-source-id: 551713a78abfe170f04e8e55318af6e157bae7da
2021-05-13 09:40:57 -07:00
Andrey Chursin
b7ec4892d9 commitcloud: fix doctest
Summary: D28351849 (2c3edb0989) broke doctest, this diff attempts to fix it

Reviewed By: quark-zju

Differential Revision: D28392077

fbshipit-source-id: 7f1f7261d8996b2c808399954fc895eccc2c1c06
2021-05-12 18:21:10 -07:00
Ron Mordechai
77b7617d4b Add myparenttags
Summary: I use tags extensively and I love them to be supported as well.

Reviewed By: asm89

Differential Revision: D28348565

fbshipit-source-id: 7d94d048b734c91e7d74a1c3efeefc87943066ad
2021-05-12 08:20:53 -07:00
Liubov Dmitrieva
297ab79a2a split full sync into 3 passes
Summary:
split full sync into 3 steps

Commit cloud by default pulls only 30 days of commits.

Users often see some of their commits are missing in their smartlog.

I discovered that most of the users know the '--full' option (`hg cloud sync --full`) but not the 'max_sync_age' config option.

So, they try --full option but it could fail due to very very old commits we haven't migrated to Mononoke.

Users often don't really need those commits but it's not nice that the whole sync run failed.

We know that at least latest 2 years of commits are present in Mononoke.

So if we split a bit how we sync with --full option works, it would at least result in partially successfully sync for the latest 2/3 years of commits.

Reviewed By: mitrandir77

Differential Revision: D28352355

fbshipit-source-id: b5bacd7d5256191528613e3c0bcbb21b0104ac3c
2021-05-11 14:00:16 -07:00
Liubov Dmitrieva
2c3edb0989 deprecate 4 commits at a time limitation for unhydrated pulls
Summary:
deprecate 4 commits at a time limitation for unhydrated pulls

This could speedify cloud join commands significantly (by many X times) and hg cloud sync --full command.

Reviewed By: farnz

Differential Revision: D28351849

fbshipit-source-id: f9f3d7a5c07d61cb51a5bb6284afaad963662c94
2021-05-11 12:40:39 -07:00
CodemodService Bot
0462f803c0 Daily arc lint --take BLACK
Reviewed By: zertosh

Differential Revision: D28282435

fbshipit-source-id: 26037c582a35a958702b5d9f5c371e19664ba3c6
2021-05-07 05:16:05 -07:00
Marc Fisher
9bdd73be2c Add stables template keyword to hg.
Summary:
Adding support for the stables template keyword in stablerev extension.

This keyword calls out to a script specified in the config stablerev.stables_cmd to get a list of stable aliases for a given revision.

Reviewed By: quark-zju

Differential Revision: D28204529

fbshipit-source-id: 3c5b21846ce6f686afddd00d3326a54b85be87dd
2021-05-06 23:52:32 -07:00
Thomas Orozco
ea8332c47d mononokepeer: increment mononoke_read_bytes on read()
Summary:
We were only incrementing this on `readline`, which resulted in very low
numbers. While in there, I also removed `self._totalbytes` as that was unused.

Reviewed By: johansglock

Differential Revision: D28260141

fbshipit-source-id: 6d9008f9342adaf75eecc8ed8c872f64212cd1f7
2021-05-06 13:41:52 -07:00
Jun Wu
f36574ef0f test-eager-exchange: test treemanifest reading
Summary:
Add a subtree to exercise treemanifest logic. Blobs in EagerRepo are verified
so we need to disable flatcompat.

Reviewed By: DurhamG

Differential Revision: D28006550

fbshipit-source-id: ac7157a9c01ed99f703601613fb3cf06add69003
2021-05-06 12:13:17 -07:00
Jun Wu
b6d24fc969 eagerepo: support "test:name" repo URLs
Summary: This makes it easier to use it in tests.

Reviewed By: DurhamG

Differential Revision: D28006549

fbshipit-source-id: 90e29b220453a3d7a260d0a62d697d64363d9a6c
2021-05-06 12:13:17 -07:00
Jun Wu
d9b901b22c extensions: always enable remotefilelog
Summary:
This makes it easier to get rid of revlog stores.

`debugindexdot` is no longer supported since it reads revlogs.
Two tests use flat manifest bundles. They are no longer supported
due to remotefilelog today has some assumptions that treemanifest
extension is also being used.

Reviewed By: DurhamG

Differential Revision: D27971126

fbshipit-source-id: fdb992a8d72bbcf562b5cb95b3f29051dd1c9464
2021-05-06 12:13:17 -07:00
Jun Wu
6763719506 eagerpeer: add clone support
Summary:
Add clone endpoint so we can clone from an eager test repo.

Note: the master group is empty and "cloneata" does not quite work yet due to
EagerRepo not writing to the master group. It will be fixed later.

Reviewed By: DurhamG

Differential Revision: D27971121

fbshipit-source-id: 0cc35136c6987673c2c4fbbd892c344c3586fcb7
2021-05-06 12:13:17 -07:00
Johan Schuijt-Li
e57f819b6e rage/debug output for schemes
Summary:
Add debug output to rage to make sure we have the relevant information in case
we need to debug issues with schemes.

Reviewed By: quark-zju

Differential Revision: D28222910

fbshipit-source-id: 9499c736d61b2c0e4568e05a3afc0ae9730acedf
2021-05-06 04:39:19 -07:00
Johan Schuijt-Li
c120560edf integrate hgext schemes into fb mercurial
Reviewed By: liubov-dmitrieva

Differential Revision: D28185505

fbshipit-source-id: 1eed74bf492c5f22abb7d616b9492eb2a549aead
2021-05-06 04:39:19 -07:00
Johan Schuijt-Li
395d04cecc import schemes hgext
Summary:
Import from upstream:
https://www.mercurial-scm.org/repo/hg/file/tip/hgext/schemes.py

at revision 47060:fde5bb5d1acf

Reviewed By: mitrandir77

Differential Revision: D28185506

fbshipit-source-id: 651398fb76adf0e05fcd22afd8f39e8f941919d5
2021-05-06 04:39:19 -07:00
Jun Wu
2cf4957de0 runtests: expand $TESTTMP in hgrc
Summary:
This makes it easier to use `--keep` to investigate tests by using
`--configfile`.

Reviewed By: kulshrax

Differential Revision: D27971122

fbshipit-source-id: 8adcbeab825155858499c24ca74c2979049adeda
2021-05-05 17:53:39 -07:00
Jun Wu
f347dc94f9 remotefilelog: do not use f strings
Summary: The syntax is not supported by Python 2.

Reviewed By: DurhamG

Differential Revision: D28233280

fbshipit-source-id: 9f882827b1357cb339e60180acadb38842c3cf8d
2021-05-05 13:30:11 -07:00
Jun Wu
2e07267f2d remotefilelog: do not use **kwargs,
Summary: The syntax is not supported by Python 2.

Reviewed By: DurhamG

Differential Revision: D28232995

fbshipit-source-id: 62058751b4f00b78a2bd56908100a7bb7a3adfde
2021-05-05 13:22:30 -07:00
Jun Wu
60e240b17e eagerepo: fix Windows compatibility
Summary: Windows path like `eagerepo:///C:\foo\bar` needs special handling.

Reviewed By: kulshrax

Differential Revision: D27971119

fbshipit-source-id: 9d4b87782eca2734b708565f0ee22a7495253cff
2021-05-05 12:01:50 -07:00
Genevieve Helsel
100fccc193 retire hg gc
Summary: `hg gc` does not do anything anymore, so in order to reduce confusion, let's just print a message that says it is no longer supported and provide a manual remediation.

Reviewed By: xavierd

Differential Revision: D28164614

fbshipit-source-id: 7ed2392cdb0091cd604a15b4c2382632706981f2
2021-05-05 11:48:58 -07:00
Jun Wu
c793852fba treemanifest: do not bundle trees present on server
Summary:
This avoids issues where the tree is stored without p1, p2. It is similar to
what we do for public commits (in createtreepackpart):

  if sendtrees == shallowbundle.AllTrees or ctx.phase() != phases.public:
      ...

Note: the trees API actually provides p1, p2, but p1, p2 are dropped when
writing to the current data store implementation.

Reviewed By: liubov-dmitrieva

Differential Revision: D28200388

fbshipit-source-id: e1fe93d8ae8576e691077d6db432d55f7b9d498d
2021-05-05 09:48:29 -07:00
Jun Wu
e687a92d10 pyedenapi: rename trees to storetrees
Summary:
The `trees` API is coupled with a store. We're going to add another API that is
not coupled with a store so let's rename `trees` to `storetrees`.

Reviewed By: liubov-dmitrieva

Differential Revision: D28200389

fbshipit-source-id: 826116f0b461873b2f5df07e7fd35e6d1018f929
2021-05-05 09:48:29 -07:00
Liubov Dmitrieva
b506eeea0c do not perform syncing for public commits
Summary: we display incorrectly 'local changes' for public commits

Reviewed By: DurhamG

Differential Revision: D28000088

fbshipit-source-id: 81bb8926dd7e96bf283eabe2b0aca84a9155fea3
2021-05-05 02:48:19 -07:00
Liubov Dmitrieva
f74c212472 remove dead code
Summary: the option has been deprecated and is not used anywhere

Reviewed By: krallin

Differential Revision: D28191314

fbshipit-source-id: f5f092b93a9644c8249628520d8d707b60854aac
2021-05-05 02:45:45 -07:00
Durham Goode
27d58bbd42 configs: delete applydynamicconfig
Summary: This is no longer used.

Reviewed By: quark-zju

Differential Revision: D26676518

fbshipit-source-id: 1cc9c32e2a95d6db78160f33ab5423338ea82e04
2021-05-04 22:18:55 -07:00
John Reese
9fd86a4fae apply upgraded black 21.4b2 formatting to fbsource
Summary:
This applies the formatting changes from black v21.4b2 to all covered
projects in fbsource. Most changes are to single line docstrings, as black
will now remove leading and trailing whitespace to match PEP8. Any other
formatting changes are likely due to files that landed without formatting,
or files that previously triggered errors in black.

Any changes to code should be AST identical. Any test failures are likely
due to bad tests, or testing against the output of pyfmt.

Reviewed By: thatch

Differential Revision: D28204910

fbshipit-source-id: 804725bcd14f763e90c5ddff1d0418117c15809a
2021-05-04 22:16:51 -07:00
Jun Wu
fa7ba86bad exchange: skip pull phase if narrow-heads is on
Summary: We skip it in other places but missed this one. Skip it too.

Reviewed By: kulshrax

Differential Revision: D27957853

fbshipit-source-id: 429d25e8b692218c9bae6c10ad76d08495a4bc66
2021-05-04 18:28:45 -07:00
Jun Wu
ae81c8fd34 edenapi: force use edenapi if ui.ssh is false
Summary: If ui.ssh is "false", then ssh cannot be used at all. Force using edenapi.

Reviewed By: kulshrax

Differential Revision: D27957312

fbshipit-source-id: 9860344779e6a6bab557d3f953ee38e40fadb78b
2021-05-04 18:28:45 -07:00
Jun Wu
66c0778db7 treemanifest: enable useruststore by default
Summary: It has been enabled for a long time in our production config.

Reviewed By: kulshrax

Differential Revision: D27953636

fbshipit-source-id: 428f6e8a3e7eae6d44c61970624a75d7d1ab3e36
2021-05-04 18:28:45 -07:00
Jun Wu
37404e7f64 treemanifest: enable ondemandfetch by default
Summary: It has been enabled for a long time in our production config.

Reviewed By: kulshrax

Differential Revision: D27953635

fbshipit-source-id: a351342fbc8cffccd16967bd0e7032ac3e4f35cf
2021-05-04 17:00:56 -07:00