Commit Graph

999 Commits

Author SHA1 Message Date
Jun Wu
c6741e4c3a Back out "smartset: back out use Rust reentrant generator for generatorset"
Summary:
The original issue was a rust-cpython bug, solved by D24698226, or https://github.com/dgrunwald/rust-cpython/pull/244.

Original commit changeset: 08f598df0892

Reviewed By: sfilipco

Differential Revision: D24759765

fbshipit-source-id: f9a1359cfce68c8754ddd1bcb8bfc54bf75af7ff
2020-11-06 16:25:00 -08:00
Durham Goode
a1d2d702d7 datastore: fix bug in indexedlogdatastore max log size
Summary:
The goal of this code was to divide the cache limit by the number of
logs. Instead it divided the cache limit by the default per-log size (2GB). That
results in a very small max-bytes-per-log so data was being thrown out
constantly. This fixes it and updates tests to actually demonstrate the issue.

Reviewed By: kulshrax

Differential Revision: D24712842

fbshipit-source-id: 8062758b5bfa40493e2003d5a9028d601b1522b1
2020-11-05 09:48:26 -08:00
Katherine Zak
dcbf01f81c rm reading "cert" key from .arcrc
Summary: This diff removes reading the token from the "cert" property in the `~/.arcrc` and forces the caller to use either an OAuth token or CATs.

Reviewed By: quark-zju

Differential Revision: D24242614

fbshipit-source-id: 18538270102b7aa28731e82c8dd21f5da9e2f2d6
2020-10-29 06:13:26 -07:00
Durham Goode
2cf0157824 debug: add --wait to debuglock
Summary:
We have automation that wants to use this to hold the lock while doing
some maintenance. They want the ability to wait for the lock so they don't have
to busy loop.

Reviewed By: snarkmaster

Differential Revision: D24604466

fbshipit-source-id: be02539908655e183f334865718b68b633b069a5
2020-10-28 19:36:17 -07:00
Durham Goode
e6af9b25b1 py3: fix globalrevs
Summary: The tests now pass with python 3.

Reviewed By: singhsrb

Differential Revision: D24583278

fbshipit-source-id: 7e0a0805751b1d3f1c77dfcd20df5b460994820a
2020-10-28 08:56:55 -07:00
Saurabh Singh
7fcc71e9d4 smartset: back out use Rust reentrant generator for generatorset
Summary: This is essentially a backout of D24365328 (9f664a8b30).

Reviewed By: DurhamG

Differential Revision: D24544495

fbshipit-source-id: 08f598df0892a8479fac563096f9782038e18dfe
2020-10-26 11:58:31 -07:00
Mateusz Kwapich
1c153d47bb make replaybookmarks py3-compatible
Reviewed By: DurhamG

Differential Revision: D24508222

fbshipit-source-id: 091ff81cae49ff7e4882edd59ace6f024e26c426
2020-10-23 12:37:01 -07:00
Jun Wu
9f664a8b30 smartset: use Rust reentrant generator for generatorset
Summary:
The generatorset has a pure Python implementation for rewindable generator.
However it is not thread-safe, and we want thread-safety since the set
iteration will be driven by async Rust in multiple threads.

One of the issues is, during thread 1 `next(gen)`, thread 2 might call
`next(gen)`, and that's not allowed by Python.

Fix them by switching to the Rust RGenerator.

Reviewed By: DurhamG

Differential Revision: D24365328

fbshipit-source-id: 2785e80c7c460a7f754ed23e3af99f4a5c9fbcdf
2020-10-21 17:17:08 -07:00
Xavier Deguillard
c62cd9e8c8 revisionstore: move the repair logic from doctor
Summary:
The ContentStore/Metadatastore are made of several different stores, attempting
to expose all of them to Python to drive the repair logic from there would leak
implementation detail of how the stores are implemented.

Instead, let's simply expose a single `repair` function out of the
pyrevisionstore crate that takes care of repairing all of the underlying
stores. For now, this is just moving code around, but a future diff will
integrate the LFS stores.

Reviewed By: DurhamG

Differential Revision: D24449203

fbshipit-source-id: 1631ced9068716453cb404bf7e65cefbf2db5247
2020-10-21 13:20:51 -07:00
Jun Wu
d86802a980 parser: add a function to walk through a tree
Summary: This will be used to test if a keyword is in an AST.

Reviewed By: DurhamG

Differential Revision: D24293499

fbshipit-source-id: c4c238e09fae6728d9d20ffe94e04194ac20bbb8
2020-10-20 16:45:43 -07:00
Jun Wu
5a6161f140 repo: unconditionally migrate to specified visibility and dirstate format
Summary:
There were 2 recent reports about "visibleheads" not being enabled.

Previously, to enable visibility, one has to run a `pull` command with
`visibility.automirate=start` and `visibility.enabled=true`, and
`pull.automirate=true`, and our current config does not have
`visibility.automigrate`.

There are other issues with pull-based auto migration. For example, the edenfs
backing repo does not run pull and won't be migrated.

Move the visibility and dirstate automigrate logic to `repo.__init__` so they
always run. Also simplify the configuration so a single config
`visibility.enabled=true` (or `format.dirstate`) is enough to trigger the
migration.

Reviewed By: DurhamG

Differential Revision: D24340586

fbshipit-source-id: 2b8ed8da51c2345c9e6e680519dbc4f10eab1ac0
2020-10-20 16:36:50 -07:00
Jun Wu
9de7c7a22d repo: drop format.usestore=false support
Summary:
We expect `.hg/store` to exist (ex. `storerequirements`). Drop support for repo
layout without a store.

Reviewed By: DurhamG

Differential Revision: D24340585

fbshipit-source-id: 757dc8633db9549a6bea082281326ebdefcbfcba
2020-10-20 16:36:50 -07:00
Jun Wu
e179789b7b tests: remove test cases about legacy dirstates
Summary:
Treestate is the only supported dirstate format. Remove test cases about legacy
dirstates.

Reviewed By: DurhamG

Differential Revision: D24340584

fbshipit-source-id: 2ebaa7d5aa9e8639e3ce4caa67c4f93155b52558
2020-10-20 16:36:50 -07:00
Jun Wu
c9d716fd51 test-rebase-emptycommit: enable segmented changelog
Reviewed By: singhsrb

Differential Revision: D24270422

fbshipit-source-id: d2dcf6db2a0c8ebd90a2edb3ee1149d8f9bd4fbe
2020-10-20 15:24:29 -07:00
Jun Wu
063bc373af test-rebase-check-restore: enable segmented changelog
Reviewed By: singhsrb

Differential Revision: D24270424

fbshipit-source-id: 2833dbbdc7ec34375f070fc6ce273265272a205e
2020-10-20 15:24:29 -07:00
Jun Wu
293d05dd40 test-rebase-conflicts: enable segmented changelog
Reviewed By: singhsrb

Differential Revision: D24258215

fbshipit-source-id: 9a9bba13c3963c79df5e672e5ac011ab7c7e5698
2020-10-20 15:24:29 -07:00
Jun Wu
20ac277bc5 test-rebase-brute-force: enable segmented changelog
Reviewed By: singhsrb

Differential Revision: D24258213

fbshipit-source-id: aceb6e0792b4646eb0805f56d981751eb85ae340
2020-10-20 15:24:29 -07:00
Jun Wu
af38cf3cb9 test-rebase-abort: enable segmented changelog
Reviewed By: singhsrb

Differential Revision: D24258214

fbshipit-source-id: db7ecb6b874da81d0e553cd9f1514ee5559635b7
2020-10-20 15:24:29 -07:00
Jun Wu
d39be1d667 tests: mark more tests as compatible with segmented changelog
Reviewed By: singhsrb

Differential Revision: D24214459

fbshipit-source-id: 7c125cbc2c0d559d337bd01107b784002e2dc9ea
2020-10-20 15:24:29 -07:00
Jun Wu
62f30ea23e changegroup: do not use bad linkrev to filter out file revisions
Summary:
If linkrevs are known unreliable, do not use them. This fixes some cases with
segmented changelog, file revisions are incorrectly filtered out and not sent
to client.

Reviewed By: DurhamG

Differential Revision: D24233228

fbshipit-source-id: 268a95b44c0217658b849317cb0a680450b91d23
2020-10-20 15:24:28 -07:00
Jun Wu
a7ba754d88 rebase: remove more uses of revnums
Summary: Replace revnum with commit hashes.

Reviewed By: singhsrb

Differential Revision: D24233229

fbshipit-source-id: 98b4d45b542b65c375cef405b45b66cd2dad22e4
2020-10-20 15:24:28 -07:00
Jun Wu
8fe79db4b1 context: do not warn broken dirstate repetitively
Summary: This makes the output less verbose.

Reviewed By: DurhamG

Differential Revision: D24411259

fbshipit-source-id: 50b175f2e31ffda62e78dd226775edaf53893bce
2020-10-20 15:24:28 -07:00
Jun Wu
79764c4e04 changelog2: track "tip" explicitly
Summary:
With segmented changelog, the maximum revision is not always the "tip".
Therefore, track "tip" explicitly in svfs / metalog. This improves
compatibility.

Reviewed By: DurhamG

Differential Revision: D24214711

fbshipit-source-id: 7b22de7ac684a90965a695924bdb9114758aae43
2020-10-20 15:24:28 -07:00
Jun Wu
5fad63b010 dag: drop last high level segment by default
Summary:
Previously, we keep the last high level segment per level in memory, and
drop it on disk. When we cross the memory / disk boundary, we had to
maintain such properties carefully. That was needed because some DAG
algorithms rely on complete high level segments.

Now that no DAG algorithms depend on such properties, let's just drop
the logic adding the last segment back to simplify the code.

This removes the need of building segments after open() and sync().

Reviewed By: sfilipco

Differential Revision: D24399515

fbshipit-source-id: 4c640d9aa03c050fcd97f70ee386e32d3a8ee26d
2020-10-20 15:19:29 -07:00
Jun Wu
0498f87da1 lock: detect deadlock in a same process
Summary:
This is for cases like:

    repo1 = hg.repository(path)
    repo2 = hg.repository(path)  # same path
    with repo1.lock():
        repo2.lock()  # deadlock

Previously it simply deadlocks with confusing message:

    waiting for lock held by process X  # X is the current process

With this change it will raise a ProgrammingError about deadlock.

Reviewed By: DurhamG

Differential Revision: D24214458

fbshipit-source-id: 57faddaed6d2f1d36fab067c2e1f63b3cda0a1fe
2020-10-19 17:07:30 -07:00
Jun Wu
9bf6b674a6 config: use Rust graph render as default
Summary: Change the legacy Python graph render to Rust renderer.

Reviewed By: DurhamG

Differential Revision: D24317802

fbshipit-source-id: 4c3dc3a6dd02b7ebe79596a8e77f4b6b139d2e20
2020-10-19 17:07:30 -07:00
Jun Wu
7b132fb6e2 smartlog: stop filtering commits by date
Summary:
The hide-before config was added to encourage people to actively hide unused
drafts for repo performance, instead of keeping unused draft forever, since a
lot of code paths assume `len(draft)` is small. See D13993584 (28b4dfbb38) for more context.

Now our hide-before data is set to 2.5 years ago (2018-2-25), this change
probably only affects a very small number of users.

Reviewed By: DurhamG

Differential Revision: D24298198

fbshipit-source-id: 938aca1222b55e09fdb058ff01bc063733f201dc
2020-10-15 22:48:41 -07:00
Durham Goode
174f05f3fb convert: disable p4 tests on Python 3
Summary:
Ideally we'd just delete the p4 convert functionality, but I'm too lazy
to go through and extract it right now.

This was recently enabled when I enabled all the convert tests.  We don't use the p4 logic, so it's safe to just turn back off to get a release out.

Reviewed By: quark-zju

Differential Revision: D24352068

fbshipit-source-id: 6f3a1f88739b2e2348aff00e8cae333473bbe71a
2020-10-15 22:23:24 -07:00
Durham Goode
9da5707131 checkout: disable fetching history during checkout
Summary:
We're seeing history fetching take quite a while during checkout and
rebase, but it's not really necessary for a checkout. In addition to it being
slow, if memcache doesn't have the history entry we'll fall back to a full
revision fetch from the server, which includes data. Let's disable prefetching
history during checkout.

Reviewed By: quark-zju

Differential Revision: D24295594

fbshipit-source-id: 70aa6e1925074b6546626a5192a7562d6da31f2b
2020-10-15 11:19:23 -07:00
John Reese
737849c9dd apply black 20.8b1 formatting update
Summary:
allow-large-files

black_any_style

Reviewed By: zertosh

Differential Revision: D24325133

fbshipit-source-id: b4afe80d1e8b2bc993f4b8e3822c02964df47462
2020-10-14 20:21:52 -07:00
Jun Wu
49dd754fa6 smartlog: do not use null as --master fallback
Summary:
Previously `--master` falls back to `null` if `-r` is provided for
compatibility. That could be surprising as smartlog's sorting does not work
well without a master branch.

Change `--master` to a consistent value regardless of `-r` is provided or not.

Reviewed By: simpkins

Differential Revision: D24297586

fbshipit-source-id: c6e695333ae17db11dcb98d9cdb57cc2f298ff64
2020-10-14 12:57:36 -07:00
Durham Goode
0fed091abb py3: fix treemanifest server tests
Summary:
simplecache didn't support binary values, which treemanifest uses.
Let's fix that.

Reviewed By: quark-zju

Differential Revision: D24290634

fbshipit-source-id: 2fd5306e10c3354d87832eb6765baca5dbb26d3e
2020-10-14 10:30:32 -07:00
Durham Goode
bff14ff181 py3: fix more tests
Summary: Some of these tests were already passing. Some required minor fixes to pass.

Reviewed By: quark-zju

Differential Revision: D24290635

fbshipit-source-id: 789d3033142466c868057da8f9f23d27aede8ea4
2020-10-14 10:30:32 -07:00
Durham Goode
0cbe0887f6 py3: fix test-convert*
Summary:
They now pass. Didn't test the p4 tests since I don't have p4
available, but I don't think we use it anymore.

Reviewed By: quark-zju

Differential Revision: D24284643

fbshipit-source-id: 184697052c0b7000ad9765b4033964b4d382b811
2020-10-14 10:30:32 -07:00
Xavier Deguillard
bc351a5891 templater: decode the result of diff()
Summary:
Template filters are expecting the output of the template to be a unicode
string, not a byte string. Thus we need to ensure that diff return a unicode
string.

Reviewed By: DurhamG

Differential Revision: D24290908

fbshipit-source-id: 70899249c3e26edd1714fd901fa6af405b61e544
2020-10-14 09:00:53 -07:00
Lukas Piatkowski
715351d576 mononoke/integration tests: fix test-clienttelemetry.t by disabling features
Summary:
The logic inside `eden/scm/tests/features.py` script enables certain
features of HG on tests based on the name of the test. Mononoke's integration
tests suit reuses eden/scm's tests suit and as a consequence it triggers the
same `features.py` logic. It was fine until D24201934 (f5988c415c) introduced a feature for
test that is named in the same way as one of the Mononoke tests - the
`test-clienttelemetry.t`.

In order to fix this problem Mononoke will pass a `--nofeatures` flag to the
`eden/scm/run-tests.py` invocations that will turn off the usage of
`features.py`.

Reviewed By: farnz

Differential Revision: D24276294

fbshipit-source-id: eb28ed55a05de3b012e37407603c2370adaaad16
2020-10-13 05:28:27 -07:00
Meyer Jacobs
f9958ca35a taggederror: introduce category and transience metadata and precedence
Summary:
This change introduces two new metadata types, Category and Transience, and a mechanism for Category to provide a default Fault and Transience, which can be overriden by the user.

Also introduces a mechanism for attempting to log exceptions which occur during exception logging, falling back to the previous behavior of just swallowing the exception on failure.

Reviewed By: DurhamG

Differential Revision: D22677565

fbshipit-source-id: 1cf75ca1e2a65964a0ede1f072439378a46bd391
2020-10-12 17:17:34 -07:00
Xavier Deguillard
437031872f fix test-import-t.py take 2
Summary: Whitespaces are needed.

Reviewed By: quark-zju

Differential Revision: D24264452

fbshipit-source-id: a7ecfbe1bba7d14f92d5218f18d55361a3f4319e
2020-10-12 15:43:57 -07:00
Jun Wu
a35f8765c0 lock: print actual lockinfo on error
Summary:
Change

  abort: repository repo: timed out waiting for lock held by <lockinfo ...>

to:

  abort: repository repo: timed out waiting for lock held by process '842210' on host 'hostname'

Reviewed By: singhsrb

Differential Revision: D24214462

fbshipit-source-id: 65056ebb9764651b2f0126061fafdfdefaa4e9c6
2020-10-12 11:22:20 -07:00
Xavier Deguillard
964e0106c9 tests: fix test-import-t.py
Summary: The rev numbers are almost gone, remove them from the test.

Reviewed By: sfilipco

Differential Revision: D24255156

fbshipit-source-id: 5cbc4a71c2d3f773c5b474d1edce84ceceb50bf9
2020-10-12 10:18:45 -07:00
Jun Wu
e4b677c6e1 remotefilelog: respect invalidatelinkrev store requirement
Summary:
With invalidatelinkrev, linkrev stored in revlog shouldn't be used.
This makes some tests pass with segmented changelog.

Reviewed By: singhsrb

Differential Revision: D24201944

fbshipit-source-id: 6473c30266c79aa97a955b1a6c867411cc67de2b
2020-10-09 13:53:18 -07:00
Jun Wu
2f1c54ae27 clone: copy modern store files
Summary:
Copy segmented changelog, metalog and mutation store for local clones.
This mainly affects tests.

Reviewed By: singhsrb

Differential Revision: D24201941

fbshipit-source-id: c60da9e2bf982a6f66004415e45178749157745e
2020-10-09 13:53:18 -07:00
Jun Wu
631034a2d6 gpg: remove revnum from output
Reviewed By: singhsrb

Differential Revision: D24201943

fbshipit-source-id: 6998ff68ba50480fb7feb4be3b35de1ea4a3ab4a
2020-10-09 13:53:17 -07:00
Jun Wu
3700c832f9 rebase: remove revnum from output
Reviewed By: singhsrb

Differential Revision: D24201945

fbshipit-source-id: 57b8fa1f98625ff186fac76009b1600f22e001bb
2020-10-09 13:53:17 -07:00
Jun Wu
38113e78ee summary: remove revnum from output
Reviewed By: singhsrb

Differential Revision: D24201936

fbshipit-source-id: a864fd1030dd8641ead46d434daadae6d4fca244
2020-10-09 13:53:17 -07:00
Durham Goode
43fe23f09d py3: fix revlog path encodings
Summary: This is needed to move our hg servers to python 3.

Reviewed By: quark-zju

Differential Revision: D24204056

fbshipit-source-id: cbaf97893f8f77b535952ac290766f0fd5e14f0c
2020-10-09 13:31:58 -07:00
Stefan Filip
befcdb1672 tests: fix test-casefolding
Summary: Broken after sweeping changes to remove revnums from tests.

Reviewed By: quark-zju

Differential Revision: D24220976

fbshipit-source-id: bd513b056bf16872ce028c835d680aaa87938917
2020-10-09 10:26:15 -07:00
Jun Wu
09c28abd20 graft: remove revnum from output
Reviewed By: singhsrb

Differential Revision: D24201937

fbshipit-source-id: bad22b741410ef990a709f77de5f843de7c3b5dc
2020-10-08 22:33:08 -07:00
Jun Wu
ec02fad2c6 bisect: remove revnum from output
Reviewed By: singhsrb

Differential Revision: D24201938

fbshipit-source-id: a6443465feac1ba43962e44a679091f655b24d16
2020-10-08 22:33:08 -07:00
Jun Wu
b01b79cf76 backout: remove revnum from output
Reviewed By: singhsrb

Differential Revision: D24201935

fbshipit-source-id: e1f4a9c9c82f401cc5b5ed9df464a58da64a0b65
2020-10-08 22:33:07 -07:00