Commit Graph

1095 Commits

Author SHA1 Message Date
Jun Wu
5ba3eac2a4 discovery: add debug message for size of initial heads
Summary: This makes it easier to reason about changes in reducing heads exchanged.

Reviewed By: DurhamG

Differential Revision: D26207589

fbshipit-source-id: 49c0c0dc25355a321c1aa4c9edfb5c43d2f23fd8
2021-02-03 20:32:42 -08:00
Durham Goode
88900aaf93 dynamicconfig: remove legacylist and disallowedlist deprecation logic
Summary:
The original migration strategy with dynamicconfig was to fix configs
one by one until the dynamicconfig values matched the chef/static ones, then we
can turn off chef/static configs. This looks to be too much work, so we're going
to try a different strategy of just turning off all chef/static configs on a
small number of hosts and seeing what breaks.

The legacylist and disallowlist configs were part of the old strategy, and they
make it more complicated to fix dynamicconfig mismatches, so let's get rid of
them.

Reviewed By: quark-zju

Differential Revision: D26208548

fbshipit-source-id: 63171f1f16aa0498c0eefa994dffaeb8e0cc0d72
2021-02-03 09:53:00 -08:00
Jun Wu
855122ad66 gracefully ignore invalid JSON in app_samples data
Summary: This would allow us to ignore bogus data and still get something useful for logging.

Reviewed By: DurhamG

Differential Revision: D20343844

fbshipit-source-id: 763d294bc44bb203c1f206ca80e0839396e8de6e
2021-02-02 16:28:21 -08:00
Durham Goode
13de9f801a doctor: repair treemanifest indexedlogs as well
Summary: Previously we only repaired the file indexedlogs.

Reviewed By: xavierd

Differential Revision: D26202423

fbshipit-source-id: b1c673ae69a357d66ab2baf5c36985a3b0597427
2021-02-02 16:21:01 -08:00
Jun Wu
59f4d938b4 revset: remove branchpoint()
Summary:
It is already broken with segmented changelog (it assumes 0..len(repo) are
valid revs). It is super slow and cannot be optimized efficiently. The _only_
non-zero-exit-code usage in the past month is like:

  hg log -r 'reverse(children(ancestors(remote/master) and branchpoint()) and draft() and age("<4d"))'

which takes 40 to 100s and can be rewritten using more efficient queries like `parents(roots(draft()))`.

Reviewed By: singhsrb

Differential Revision: D26158011

fbshipit-source-id: 7957710f27af8a83920021a228e4fa00439b6f3d
2021-01-29 20:21:29 -08:00
Jun Wu
358466ee0f transaction: record transaction name to metalog
Summary:
For `repo.transaction("tr-name")`, this records `Transaction: tr-name` to
metalog commit description.

It can be helpful to narrow things down for commands with multiple
transactions.

In the future we might want to attach more data to the logging (ex. what the
commit cloud local, remote states at the time of syncing). However I didn't
do it now since metalog is designed to hold repository data, not too much
logging data. With a better logging infra we might want to move `config` out
from metalog, associated with metalog root ids.

Reviewed By: DurhamG

Differential Revision: D25984805

fbshipit-source-id: 59c074272cff555c6ff11dd755f7e3ce9a292eb6
2021-01-29 12:36:08 -08:00
Durham Goode
7f555d2d06 http: improve error messages from http failures
Summary:
Currently the data layer eats all errors from remote stores and treats
them as KeyErrors. This hides connection issues from users behind obscure
KeyErrors. Let's make it so that any non-key error reported by the remote store
is propagated up as a legitimate error.

This diff makes Http errors from EdenApi show up with a nicer error message,
suggesting that the user run fixmyserver.

Further fixes will probably be necessary to categorize other errors from the
remote store more nicely.

Reviewed By: quark-zju

Differential Revision: D26117726

fbshipit-source-id: 7d7dee6ec101c6a1d226185bb27423d977096050
2021-01-29 09:40:19 -08:00
Liubov Dmitrieva
339788a95e fix tests
Summary: Fix tests broken by the deprecation notice D26073439 (14a0d96b04)

Reviewed By: johansglock

Differential Revision: D26099594

fbshipit-source-id: 65b973c4f1dcc7a032e747c12ce59a1d0c04091e
2021-01-27 05:12:09 -08:00
Liubov Dmitrieva
14a0d96b04 add deprecation notice for pullbackup related set of commands
Summary:
this is needed to remind users to use the commands related to the Commit Cloud Workspaces

This is also needed to avoid confusion - the current error is not very informative:

```
connected to twshared9522.24.frc3.facebook.com session Wfq9tGuoPSBJH8nc
abort: 'listkeyspatterns' command is not supported for the server ssh://hg.vip.facebook.com//data/scm/fbsource?infinitepush_write
```

Reviewed By: markbt

Differential Revision: D26073439

fbshipit-source-id: 58ee477bc021a0796e22cc18610225ee003e06be
2021-01-26 08:21:07 -08:00
Liubov Dmitrieva
2c15a29f0f fix unit test that was broken by D19803760
Summary: fix unit test that was broken by D19803760 (8e35bcc49d) to unblock release

Reviewed By: krallin

Differential Revision: D26073002

fbshipit-source-id: 88da60f5806ef5cb2f832922e503309836542e09
2021-01-26 04:33:47 -08:00
Jan Mazur
d1b291d72a flush stream to always print out interactive prompt
Summary: Suddenly prompt stopped appearing for me. Flush the stream to be sure that it's printed out.

Reviewed By: HarveyHunt

Differential Revision: D25956018

fbshipit-source-id: 83419037fa6ce672e203385b71f1403a738d0c90
2021-01-22 07:46:11 -08:00
Liubov Dmitrieva
a1dd7229e6 dynamic config: remove unnecessary option from the dynamic config
Summary: this infinitepush option is true by default

Reviewed By: DurhamG

Differential Revision: D25995928

fbshipit-source-id: 0c1d50a96dc16ee4d8c45da4f4a213baa49bb8a4
2021-01-22 06:24:23 -08:00
Liubov Dmitrieva
9bd3235f3f deprecate owner_team option in favour of ui.supportcontact
Summary:
The both options have basically the same value.

This is my next step for resolving mismatches between CC dynamic config and the current configuration and generally improving CC configuration.

Reviewed By: DurhamG

Differential Revision: D25973556

fbshipit-source-id: aae21efcd5174ed58efcb9e5d8c85831d35777ea
2021-01-20 14:15:54 -08:00
Jun Wu
f11d555a3a fsmonitor: add threshold of nonnormal files to update
Summary:
Previously, the fsmonitor state update logic will skip updating treestate if the wlock
cannot be obtained. D17468790 (8d4d0a66a2) made it wait for wlock for the painful "watchman fresh
instance" case. But things can still suck if it's not a "fresh instance" but there are just
too many nonnormal files.

This diff makes it that exceeding a threshold of nonnormal files will trigger a fsmonitor
state write as an attempt to to reduce the number of nonnormal files. In additional,
`--debug` was changed to print more internal states for debugging.

This would hopefully address issues where people have a large "nonnormal"
treestate, suffers from the bad performance issue and cannot recover from it
automatically.

Reviewed By: DurhamG

Differential Revision: D25794083

fbshipit-source-id: 741426cf31484d9318f9cfcab11d38da33ab5067
2021-01-12 16:26:48 -08:00
Stefan Filip
a9a2610cf8 tests: remove test-hgext-repogenerator.t
Summary: Unstable test.

Reviewed By: quark-zju

Differential Revision: D25879146

fbshipit-source-id: 09e41fc1e43644bba50f713a095e8a4b8c4f520a
2021-01-11 17:58:02 -08:00
Jun Wu
027709fa36 pull: use the new repo.pull API if possible
Summary:
The `pull` commmand has a lot of tech debt (with issues like inefficiency, race
conditions, named branches, etc). The new `repo.pull` API isn't designed to
support all legacy usecases.

This diff switches a subset of `pull` command that the new API can support to
the new API. It should solve race condition or `visibility.add` inefficiency
issues (ex. 20s on addgroup, 187s on visibility.add - P154836357).

Logic related to remotenames was ported from the remotenames extension.
The selectivepull tests seem strong enough to check their behaviors.

The new pull API is used by commit cloud for many months. So I turned the
new code path on by default. It can be turned off by changing
`commands.new-pull`.

There are a few test changes. The new behavior seems more correct overall:
- test-commitcloud-switch-workspace.t
  - "M/F" swap is caused by rev number: both are correct
  - "S: public remote/stable" is more correct than "S: draft"
- test-commitcloud-sync-rb-deletion.t
  - "draft1: draft remote/scratch/draft1" is more correct because
    "remote/scratch/draft1" is listed in the "book --list-subs"
    command above.
- test-commitcloud-sync-rb-enabling.t
  - "public1: draft" not showing up is more correct.
- test-fb-hgext-remotefilelog-lfs-prefetch.t
  - Difference seems to be caused by rev order.

Reviewed By: markbt

Differential Revision: D25562089

fbshipit-source-id: ac22b2f0492ab53517d580d706dfbc823fd0e0cc
2021-01-11 13:28:15 -08:00
Thomas Orozco
9d7b0267dd revisionstore: pass client correlator
Summary:
We used to get those in the old (Python) LFS extension, but didn't have them in
the new one. However, this is helpful to correlate requests to LFS with data in
hg logs. It's also convenient to be able to identify whether a set of requests
are part of the same session or not.

This diffs threads the client correlator through to the LFS store from the
Python, similarly to how it's done in EdenAPI.

Reviewed By: DurhamG

Differential Revision: D25804930

fbshipit-source-id: a5d5508617fa4184344834bbd8e3423816aa7668
2021-01-11 10:46:20 -08:00
Mark Juggurnauth-Thomas
a7e8a2c5c2 crdump: add manifest_node to crdump output
Summary:
Add `manifest_node` to the crdump output, which is the root manifest node of the commit.

This is useful for detecting commits that have the same tree content but
different metadata (e.g. if only the commit message has been edited).

Reviewed By: singhsrb

Differential Revision: D25782674

fbshipit-source-id: dfdf426833533140b676eee82e123a0cba23c77a
2021-01-11 07:30:35 -08:00
Durham Goode
9c3d099697 tests: glob out times in tests
Summary: These globs were lost as part of D25315954 (ec0b533381).

Reviewed By: quark-zju

Differential Revision: D25814934

fbshipit-source-id: b1896893e37e355a73eb136758f8966666e0ec05
2021-01-07 16:13:31 -08:00
Liubov Dmitrieva
676063f0f2 allow switching workspaces with uncommitted changes unless move to a public root is impossible
Summary:
Unless we can't update to a public root, there is nothing wrong with having local changes and switching workspaces feature.
Those are not related. Uncommited changes shouldn't impact switching workspaces.

Reviewed By: mitrandir77

Differential Revision: D25802406

fbshipit-source-id: 3fcb70864002bed11ad32621947294f643ca1fc3
2021-01-07 06:20:23 -08:00
Liubov Dmitrieva
713faa71cd allow hg cloud switch command to work for clean disconnected repos
Summary:
This was a request from users. Repo could go into a disconnected state, for example, if rejoin in fbclone fails due to some reason.
In this case it was confusing that `hg cloud switch` command doesn't work. Users have to run `hg cloud join` command first.

If the repo is disconnected but doesn't contain any relevant local changes for commit cloud, it should be fine to switch workspace.

Reviewed By: mitrandir77

Differential Revision: D25802193

fbshipit-source-id: 3216a10c3438463773602b2dfd13740866fb5908
2021-01-06 13:47:20 -08:00
Liubov Dmitrieva
19b20b43a1 enable switching from a draft commit possible for most of cases
Summary:
enable switching from a draft commit possible for most of the cases

make it possible if the public root of the current commit is an ancestor of the main bookmark

this condition we need because the remote bookmarks can be different for different workspaces and they define phases

I think it will cover most of workflows

Reviewed By: mitrandir77

Differential Revision: D25780999

fbshipit-source-id: b1c25b29a7668d51244ca43d6b0c30fa2fc068d9
2021-01-06 01:59:16 -08:00
Jun Wu
78005ae5a5 py3: fix test-run-tests.py
Summary:
When running `python3 run-tests.py test-run-tests.py`, some bytes were printed
with `b` prefix.  Convert them to `str`.

Reviewed By: DurhamG

Differential Revision: D25642164

fbshipit-source-id: f1103b24ad88d0d024f6be546bf632141f06ebd1
2020-12-18 10:38:19 -08:00
Durham Goode
cfd7ccc828 configs: add Python support for allowed_locations list
Summary:
configs.allowedlocations restricts what configs can be loaded to a
certain set of files. This will enable us to deprecate all old config locations.

This diff adds Python support and a high level test.

Reviewed By: quark-zju

Differential Revision: D25539736

fbshipit-source-id: fa2544379b65672227e0d9cf08dad7016d6bbac8
2020-12-17 06:37:54 -08:00
Jun Wu
63bb40f9c4 ui: move push/pull messages to stderr
Summary:
Those messages like "pulling from ...", "added n commits ..." belong to stderr.

This makes it possible for us to turn on verbose output for auto pull, without
breaking tools that parses stdout.

Reviewed By: sfilipco

Differential Revision: D25315955

fbshipit-source-id: 933f631610840eb5f603ad817f7560c78b19e4ad
2020-12-16 20:12:04 -08:00
Jun Wu
ec0b533381 ui: make ui.debug write to stderr
Summary:
Debug output belongs to stderr.

This makes it possible to turn on debug output without breaking programs
parsing stdout.

Reviewed By: singhsrb

Differential Revision: D25315954

fbshipit-source-id: c7813a824fbf6640cb5b80b5ed2d947e7059d53e
2020-12-15 12:07:47 -08:00
Jun Wu
d7f7bc0181 smartlog: always show "."
Summary:
With `collapse-obsolete`, `.` can be obsoleted and in the middle of a stack and
not shown up. That can be confusing. Make the smartlog revset always show the
`heads` passed in. If `.` is in `heads` (the default), then show it.

Reviewed By: DurhamG

Differential Revision: D24696595

fbshipit-source-id: 7deab109d0e0ae5e703928252bc63312d936955f
2020-12-15 11:52:02 -08:00
Jun Wu
1a1b3a3cb5 smartset: make repo required for baseset
Reviewed By: sfilipco

Differential Revision: D24365401

fbshipit-source-id: 4d0ee6d27717c1aa966086af68492295aa6ed372
2020-12-14 13:12:42 -08:00
Jun Wu
d55eae5fdb tests: remove test-extension.t
Summary:
The test is one of the most fragile tests (ex. it has traceback that is
fragile).  Since we don't support 3rd party extensions the test is less
meaningful. Let's remove it.

Reviewed By: singhsrb

Differential Revision: D25514727

fbshipit-source-id: cc3491597ba38ad84c4c2b69a275da6505ae2207
2020-12-14 13:12:41 -08:00
Jun Wu
90f40e79bd tests: mark test-atexit-epipe.t as chg incompatible
Summary:
It has a high chance of failing with:

```
   $ hg --config extensions.b=b.py --config 'pager.pager=head -1' init foo2
+  close failed in file object destructor:
+  IOError: [Errno 9] Bad file descriptor
   line1
```

when running with chg together with other tests.

Reviewed By: singhsrb

Differential Revision: D25514728

fbshipit-source-id: d9d2adea97bc6a436341be6f6d9aac43cb78c90a
2020-12-14 13:12:41 -08:00
Durham Goode
2333b25e75 tests: fix rust store test on windows
Summary: Looks like the permissions are different there. Let's glob it out.

Reviewed By: singhsrb

Differential Revision: D25507359

fbshipit-source-id: 6a5c19e41879798b829d9b6e79eba3009249c20c
2020-12-14 06:53:24 -08:00
Stanislau Hlebik
cbe07f0d17 infinitepush: select infinitepush path only for scratch bookmarks
Summary:
At the moment "hg pull -B bookmark" always fetches from infinitepush path even
if we do something like "hg pull -B master".

Let's fetch from infinitepush only if a bookmark matches scratch mather

Reviewed By: markbt

Differential Revision: D25460577

fbshipit-source-id: 6563dcd3423c6a7a70ea1c1f7acdaf5db5e21875
2020-12-13 23:48:22 -08:00
Durham Goode
d7af02abb1 revisionstore: fix paths for local indexedlog
Summary:
There was a bug with local-data indexedlog storage where it
wasn't applying the appropriate suffix, so tree data was being stored in
.hg/store/indexedlogdatastore just like file data. Let's fix that and add a
test.

Reviewed By: quark-zju

Differential Revision: D25469917

fbshipit-source-id: 731252f924f9a8014867fc077a7ef10ac9870170
2020-12-11 06:58:27 -08:00
Jun Wu
2e7f12ab4c template: fix ifcontains(rev, revset('.'), ...) compatibility with f64 compatibility
Summary:
The f64 compatibility added by D25079001 (e91552fefe) changes the `{rev}` template output.
However, the revset object is not aware of the rev mapping so the following
pattern will stop working:

  ifcontains(rev, revset('.'), '@', 'o')
  # rev: mapped, but not contains in `revset('.')`

Fix it by teaching `ifcontains(a, b, ...)` to disable f64 compatibility when
evaluating `a`, since `a` is not going to be printed out.

This fixes an issue in VSCode ISL that "You are here" is missing on non-master
commits.

Reviewed By: singhsrb

Differential Revision: D25404673

fbshipit-source-id: 3e53a2ce1f135f8825c195c5a3061dad0359c4b2
2020-12-09 10:36:28 -08:00
Xavier Deguillard
77f3f239a2 store: replace use of ctreemanifest with small manifest parser
Summary:
The code still took a dependency on Mercurial's old manifest code to parse
manifests. It turns out the manifests have a very simple format that we could
parse directly.

This avoids various copies, conversions, std::list, removes ~1k lines of code,
at the expense of adding ~100 lines of code (some of them being C++
boilerplate).

Reviewed By: fanzeyi

Differential Revision: D25385018

fbshipit-source-id: 90d4cda2b7797584bc48c086d5592a7ecaa05dfc
2020-12-09 08:40:38 -08:00
Durham Goode
4f611364eb revisionstore: add repair support to IndexedLogHgIdDatatore
Summary:
We temporarily dropped repair support when transitioning to using Store instead
of a raw RotateLog. Let's add that back now.

Reviewed By: xavierd

Differential Revision: D25371622

fbshipit-source-id: e28fc425a6ffb50c93540672b0df75a172ebbe9c
2020-12-09 07:07:40 -08:00
Durham Goode
925e457ae7 revisionstore: allow persistent IndexedLogData
Summary:
In a future diff we'll use the indexedlog stores for local data. We
want those to exist forever, so let's move IndexedLogHgIdDataStore to use a
Store under the hood, and add an enum for distinguishing between the two types
at creation time.

Reviewed By: xavierd

Differential Revision: D23915622

fbshipit-source-id: 296cf6dfcd53e5cf1ae7624fdccedf0a60a77f22
2020-12-09 07:07:40 -08:00
Stefan Filip
b991370c3f commands: add debugsegmentclone
Summary: Prototype for cloning a repository using segmented changelog.

Reviewed By: quark-zju

Differential Revision: D24954821

fbshipit-source-id: 5ff42a5d60b6ba3e85dd4eab151e902bd4b96243
2020-12-01 09:59:27 -08:00
Durham Goode
bdaa63386d fsmonitor: filter out non-utf8 files
Summary:
On Python 3 we only support utf8 files. Python 3 has a way of
representing non-utf8 strings in utf8 format by utilizing surrogateescape, but
these strings cause issues in other parts of the codebase that don't expect
suorrageescaped strings (like the Rust code). Since we simply do not support
these paths, let's filter them out as soon as we get them from Watchman.

Reviewed By: quark-zju

Differential Revision: D25134079

fbshipit-source-id: 8be6893a6114b816097422f4469ac317fa3795d1
2020-11-30 10:22:13 -08:00
Durham Goode
132f37fa14 py3: fix hggit tests
Summary:
It turns out the hggit tests weren't passing in Python 3, despite us
having removed them from our py3-broken list. Woops. This fixes them and enables
the tests.

Reviewed By: sfilipco

Differential Revision: D25095189

fbshipit-source-id: acffca34b0d5defa7575ede60621ca2ce0a2afe4
2020-11-19 18:47:02 -08:00
Jun Wu
e91552fefe template: add rev f64 compatibility
Summary:
With segmented changelog, rev can exceed f64 safe range
(Number.MAX_SAFE_INTEGER in Javascript, 9007199254740991, 0x1fffffffffffff).
If rev is used in JSON, then the JSON cannot be parsed with precise rev
information.

This diff adds a compatibility mode so template will map the out-of-range revs
to safe range, and the mapped revs can be detected, and mapped back to be
resolved correctly.

Reviewed By: sfilipco

Differential Revision: D25079001

fbshipit-source-id: 52a5a2c8345a76355b8bf81435034d90c7c31610
2020-11-19 12:15:36 -08:00
Stanislau Hlebik
0dce99888f cmdutil: fix amending a reverted file
Summary:
We had a bug: if two files were reverted and then we try to amend one of them
mercurial will actually amend both of them.

Looks like the problem was caused by "Prune files reverted by amend" block.
Previously this block was considering every file that was changed in a commit we
are about to amend and comparing with working copy. If a file is the same in a
commit we are about to amend and in the working copy then it will amended as
well.

This diff attempts to fix it by considering only files that were selected for
amending.

Reviewed By: DurhamG

Differential Revision: D24949727

fbshipit-source-id: cf6cb95af3f67ec769e8a58db3b829945133b830
2020-11-16 02:58:24 -08:00
Stanislau Hlebik
daa28549c4 cmdutil - add a test that shows invalid amend behaviour
Summary:
We have an edge case - if we reverted two files and then we try to amend only a
single one  then both of them will be amended.

This diff adds a test for this and other edge-cases. The next diff will fix it

Reviewed By: DurhamG

Differential Revision: D24949726

fbshipit-source-id: c5c53de1d67f161efa8564f89127e61ac2f28ac9
2020-11-16 02:58:23 -08:00
Thomas Orozco
3a7014ed80 streamclone: send files that might be un-inlined soon first
Summary:
When we streamclone, we snapshot the revlogs under a lock, then we start
sending. That works fine, unless we have a file whose size changes during
the sending phase. This can happen if it's promoted from a single `.i` to a
`.i` and a `.d`.

When that happens, the clone fails (the client reports it received a bunch of
corrupted data because it starts interpreting parts of files as inputs). Since
the breakage is also confusing client side, I updated the server side to also
assert that it's sending what it thinks it's sending.

Reviewed By: DurhamG

Differential Revision: D24958885

fbshipit-source-id: a0349c651b7cb63ab27546adf9944e7fba63a95d
2020-11-14 08:02:21 -08:00
Durham Goode
93ae8ba357 sparse: fix xoring of None
Summary:
We got a [report](https://fb.workplace.com/groups/scm/permalink/3379140858802177/) that a new hg build fails with an error because it can't xor None types.

```
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: PyErr {
ptype: <type 'exceptions.TypeError'>, pvalue: Some("unsupported operand type(s)
for ^: 'NoneType' and 'NoneType'"), ptraceback: Some(<traceback object at
0x00000249BB158248>) }',
```

Full stack trace is here
{P149395441}

This seems likely to be related to the diff I landed recently - D24725902 (7b1798be37).
However it's unclear why it was affecting only windows because I couldn't repro
it on linux.

Turned out that we have experimental.treematcher option disabled on windows,
which causes it to use includematcher instead of treematcher. And includematcher
returns either None or BytesMatchObject and they are impossible to xor.

This diff fixes it by converting them value to bool first, and also it adds a
test for it.

Reviewed By: singhsrb

Differential Revision: D24918192

fbshipit-source-id: 1359e8b97d26d3b1a4795b7b3d4cfa3d6d4ae843
2020-11-12 17:34:07 -08:00
John Reese
d13e0b137f apply pyfmt with usort to opted-in sources
Reviewed By: zertosh

Differential Revision: D24880203

fbshipit-source-id: 2034cdfc2712209e86d3d05c119c58f979b05c52
2020-11-10 21:25:54 -08:00
Jun Wu
c6741e4c3a Back out "smartset: back out use Rust reentrant generator for generatorset"
Summary:
The original issue was a rust-cpython bug, solved by D24698226, or https://github.com/dgrunwald/rust-cpython/pull/244.

Original commit changeset: 08f598df0892

Reviewed By: sfilipco

Differential Revision: D24759765

fbshipit-source-id: f9a1359cfce68c8754ddd1bcb8bfc54bf75af7ff
2020-11-06 16:25:00 -08:00
Durham Goode
a1d2d702d7 datastore: fix bug in indexedlogdatastore max log size
Summary:
The goal of this code was to divide the cache limit by the number of
logs. Instead it divided the cache limit by the default per-log size (2GB). That
results in a very small max-bytes-per-log so data was being thrown out
constantly. This fixes it and updates tests to actually demonstrate the issue.

Reviewed By: kulshrax

Differential Revision: D24712842

fbshipit-source-id: 8062758b5bfa40493e2003d5a9028d601b1522b1
2020-11-05 09:48:26 -08:00
Katherine Zak
dcbf01f81c rm reading "cert" key from .arcrc
Summary: This diff removes reading the token from the "cert" property in the `~/.arcrc` and forces the caller to use either an OAuth token or CATs.

Reviewed By: quark-zju

Differential Revision: D24242614

fbshipit-source-id: 18538270102b7aa28731e82c8dd21f5da9e2f2d6
2020-10-29 06:13:26 -07:00
Durham Goode
2cf0157824 debug: add --wait to debuglock
Summary:
We have automation that wants to use this to hold the lock while doing
some maintenance. They want the ability to wait for the lock so they don't have
to busy loop.

Reviewed By: snarkmaster

Differential Revision: D24604466

fbshipit-source-id: be02539908655e183f334865718b68b633b069a5
2020-10-28 19:36:17 -07:00
Durham Goode
e6af9b25b1 py3: fix globalrevs
Summary: The tests now pass with python 3.

Reviewed By: singhsrb

Differential Revision: D24583278

fbshipit-source-id: 7e0a0805751b1d3f1c77dfcd20df5b460994820a
2020-10-28 08:56:55 -07:00
Saurabh Singh
7fcc71e9d4 smartset: back out use Rust reentrant generator for generatorset
Summary: This is essentially a backout of D24365328 (9f664a8b30).

Reviewed By: DurhamG

Differential Revision: D24544495

fbshipit-source-id: 08f598df0892a8479fac563096f9782038e18dfe
2020-10-26 11:58:31 -07:00
Mateusz Kwapich
1c153d47bb make replaybookmarks py3-compatible
Reviewed By: DurhamG

Differential Revision: D24508222

fbshipit-source-id: 091ff81cae49ff7e4882edd59ace6f024e26c426
2020-10-23 12:37:01 -07:00
Jun Wu
9f664a8b30 smartset: use Rust reentrant generator for generatorset
Summary:
The generatorset has a pure Python implementation for rewindable generator.
However it is not thread-safe, and we want thread-safety since the set
iteration will be driven by async Rust in multiple threads.

One of the issues is, during thread 1 `next(gen)`, thread 2 might call
`next(gen)`, and that's not allowed by Python.

Fix them by switching to the Rust RGenerator.

Reviewed By: DurhamG

Differential Revision: D24365328

fbshipit-source-id: 2785e80c7c460a7f754ed23e3af99f4a5c9fbcdf
2020-10-21 17:17:08 -07:00
Xavier Deguillard
c62cd9e8c8 revisionstore: move the repair logic from doctor
Summary:
The ContentStore/Metadatastore are made of several different stores, attempting
to expose all of them to Python to drive the repair logic from there would leak
implementation detail of how the stores are implemented.

Instead, let's simply expose a single `repair` function out of the
pyrevisionstore crate that takes care of repairing all of the underlying
stores. For now, this is just moving code around, but a future diff will
integrate the LFS stores.

Reviewed By: DurhamG

Differential Revision: D24449203

fbshipit-source-id: 1631ced9068716453cb404bf7e65cefbf2db5247
2020-10-21 13:20:51 -07:00
Jun Wu
d86802a980 parser: add a function to walk through a tree
Summary: This will be used to test if a keyword is in an AST.

Reviewed By: DurhamG

Differential Revision: D24293499

fbshipit-source-id: c4c238e09fae6728d9d20ffe94e04194ac20bbb8
2020-10-20 16:45:43 -07:00
Jun Wu
5a6161f140 repo: unconditionally migrate to specified visibility and dirstate format
Summary:
There were 2 recent reports about "visibleheads" not being enabled.

Previously, to enable visibility, one has to run a `pull` command with
`visibility.automirate=start` and `visibility.enabled=true`, and
`pull.automirate=true`, and our current config does not have
`visibility.automigrate`.

There are other issues with pull-based auto migration. For example, the edenfs
backing repo does not run pull and won't be migrated.

Move the visibility and dirstate automigrate logic to `repo.__init__` so they
always run. Also simplify the configuration so a single config
`visibility.enabled=true` (or `format.dirstate`) is enough to trigger the
migration.

Reviewed By: DurhamG

Differential Revision: D24340586

fbshipit-source-id: 2b8ed8da51c2345c9e6e680519dbc4f10eab1ac0
2020-10-20 16:36:50 -07:00
Jun Wu
9de7c7a22d repo: drop format.usestore=false support
Summary:
We expect `.hg/store` to exist (ex. `storerequirements`). Drop support for repo
layout without a store.

Reviewed By: DurhamG

Differential Revision: D24340585

fbshipit-source-id: 757dc8633db9549a6bea082281326ebdefcbfcba
2020-10-20 16:36:50 -07:00
Jun Wu
e179789b7b tests: remove test cases about legacy dirstates
Summary:
Treestate is the only supported dirstate format. Remove test cases about legacy
dirstates.

Reviewed By: DurhamG

Differential Revision: D24340584

fbshipit-source-id: 2ebaa7d5aa9e8639e3ce4caa67c4f93155b52558
2020-10-20 16:36:50 -07:00
Jun Wu
c9d716fd51 test-rebase-emptycommit: enable segmented changelog
Reviewed By: singhsrb

Differential Revision: D24270422

fbshipit-source-id: d2dcf6db2a0c8ebd90a2edb3ee1149d8f9bd4fbe
2020-10-20 15:24:29 -07:00
Jun Wu
063bc373af test-rebase-check-restore: enable segmented changelog
Reviewed By: singhsrb

Differential Revision: D24270424

fbshipit-source-id: 2833dbbdc7ec34375f070fc6ce273265272a205e
2020-10-20 15:24:29 -07:00
Jun Wu
293d05dd40 test-rebase-conflicts: enable segmented changelog
Reviewed By: singhsrb

Differential Revision: D24258215

fbshipit-source-id: 9a9bba13c3963c79df5e672e5ac011ab7c7e5698
2020-10-20 15:24:29 -07:00
Jun Wu
20ac277bc5 test-rebase-brute-force: enable segmented changelog
Reviewed By: singhsrb

Differential Revision: D24258213

fbshipit-source-id: aceb6e0792b4646eb0805f56d981751eb85ae340
2020-10-20 15:24:29 -07:00
Jun Wu
af38cf3cb9 test-rebase-abort: enable segmented changelog
Reviewed By: singhsrb

Differential Revision: D24258214

fbshipit-source-id: db7ecb6b874da81d0e553cd9f1514ee5559635b7
2020-10-20 15:24:29 -07:00
Jun Wu
d39be1d667 tests: mark more tests as compatible with segmented changelog
Reviewed By: singhsrb

Differential Revision: D24214459

fbshipit-source-id: 7c125cbc2c0d559d337bd01107b784002e2dc9ea
2020-10-20 15:24:29 -07:00
Jun Wu
62f30ea23e changegroup: do not use bad linkrev to filter out file revisions
Summary:
If linkrevs are known unreliable, do not use them. This fixes some cases with
segmented changelog, file revisions are incorrectly filtered out and not sent
to client.

Reviewed By: DurhamG

Differential Revision: D24233228

fbshipit-source-id: 268a95b44c0217658b849317cb0a680450b91d23
2020-10-20 15:24:28 -07:00
Jun Wu
a7ba754d88 rebase: remove more uses of revnums
Summary: Replace revnum with commit hashes.

Reviewed By: singhsrb

Differential Revision: D24233229

fbshipit-source-id: 98b4d45b542b65c375cef405b45b66cd2dad22e4
2020-10-20 15:24:28 -07:00
Jun Wu
8fe79db4b1 context: do not warn broken dirstate repetitively
Summary: This makes the output less verbose.

Reviewed By: DurhamG

Differential Revision: D24411259

fbshipit-source-id: 50b175f2e31ffda62e78dd226775edaf53893bce
2020-10-20 15:24:28 -07:00
Jun Wu
79764c4e04 changelog2: track "tip" explicitly
Summary:
With segmented changelog, the maximum revision is not always the "tip".
Therefore, track "tip" explicitly in svfs / metalog. This improves
compatibility.

Reviewed By: DurhamG

Differential Revision: D24214711

fbshipit-source-id: 7b22de7ac684a90965a695924bdb9114758aae43
2020-10-20 15:24:28 -07:00
Jun Wu
5fad63b010 dag: drop last high level segment by default
Summary:
Previously, we keep the last high level segment per level in memory, and
drop it on disk. When we cross the memory / disk boundary, we had to
maintain such properties carefully. That was needed because some DAG
algorithms rely on complete high level segments.

Now that no DAG algorithms depend on such properties, let's just drop
the logic adding the last segment back to simplify the code.

This removes the need of building segments after open() and sync().

Reviewed By: sfilipco

Differential Revision: D24399515

fbshipit-source-id: 4c640d9aa03c050fcd97f70ee386e32d3a8ee26d
2020-10-20 15:19:29 -07:00
Jun Wu
0498f87da1 lock: detect deadlock in a same process
Summary:
This is for cases like:

    repo1 = hg.repository(path)
    repo2 = hg.repository(path)  # same path
    with repo1.lock():
        repo2.lock()  # deadlock

Previously it simply deadlocks with confusing message:

    waiting for lock held by process X  # X is the current process

With this change it will raise a ProgrammingError about deadlock.

Reviewed By: DurhamG

Differential Revision: D24214458

fbshipit-source-id: 57faddaed6d2f1d36fab067c2e1f63b3cda0a1fe
2020-10-19 17:07:30 -07:00
Jun Wu
9bf6b674a6 config: use Rust graph render as default
Summary: Change the legacy Python graph render to Rust renderer.

Reviewed By: DurhamG

Differential Revision: D24317802

fbshipit-source-id: 4c3dc3a6dd02b7ebe79596a8e77f4b6b139d2e20
2020-10-19 17:07:30 -07:00
Jun Wu
7b132fb6e2 smartlog: stop filtering commits by date
Summary:
The hide-before config was added to encourage people to actively hide unused
drafts for repo performance, instead of keeping unused draft forever, since a
lot of code paths assume `len(draft)` is small. See D13993584 (28b4dfbb38) for more context.

Now our hide-before data is set to 2.5 years ago (2018-2-25), this change
probably only affects a very small number of users.

Reviewed By: DurhamG

Differential Revision: D24298198

fbshipit-source-id: 938aca1222b55e09fdb058ff01bc063733f201dc
2020-10-15 22:48:41 -07:00
Durham Goode
174f05f3fb convert: disable p4 tests on Python 3
Summary:
Ideally we'd just delete the p4 convert functionality, but I'm too lazy
to go through and extract it right now.

This was recently enabled when I enabled all the convert tests.  We don't use the p4 logic, so it's safe to just turn back off to get a release out.

Reviewed By: quark-zju

Differential Revision: D24352068

fbshipit-source-id: 6f3a1f88739b2e2348aff00e8cae333473bbe71a
2020-10-15 22:23:24 -07:00
Durham Goode
9da5707131 checkout: disable fetching history during checkout
Summary:
We're seeing history fetching take quite a while during checkout and
rebase, but it's not really necessary for a checkout. In addition to it being
slow, if memcache doesn't have the history entry we'll fall back to a full
revision fetch from the server, which includes data. Let's disable prefetching
history during checkout.

Reviewed By: quark-zju

Differential Revision: D24295594

fbshipit-source-id: 70aa6e1925074b6546626a5192a7562d6da31f2b
2020-10-15 11:19:23 -07:00
John Reese
737849c9dd apply black 20.8b1 formatting update
Summary:
allow-large-files

black_any_style

Reviewed By: zertosh

Differential Revision: D24325133

fbshipit-source-id: b4afe80d1e8b2bc993f4b8e3822c02964df47462
2020-10-14 20:21:52 -07:00
Jun Wu
49dd754fa6 smartlog: do not use null as --master fallback
Summary:
Previously `--master` falls back to `null` if `-r` is provided for
compatibility. That could be surprising as smartlog's sorting does not work
well without a master branch.

Change `--master` to a consistent value regardless of `-r` is provided or not.

Reviewed By: simpkins

Differential Revision: D24297586

fbshipit-source-id: c6e695333ae17db11dcb98d9cdb57cc2f298ff64
2020-10-14 12:57:36 -07:00
Durham Goode
0fed091abb py3: fix treemanifest server tests
Summary:
simplecache didn't support binary values, which treemanifest uses.
Let's fix that.

Reviewed By: quark-zju

Differential Revision: D24290634

fbshipit-source-id: 2fd5306e10c3354d87832eb6765baca5dbb26d3e
2020-10-14 10:30:32 -07:00
Durham Goode
bff14ff181 py3: fix more tests
Summary: Some of these tests were already passing. Some required minor fixes to pass.

Reviewed By: quark-zju

Differential Revision: D24290635

fbshipit-source-id: 789d3033142466c868057da8f9f23d27aede8ea4
2020-10-14 10:30:32 -07:00
Durham Goode
0cbe0887f6 py3: fix test-convert*
Summary:
They now pass. Didn't test the p4 tests since I don't have p4
available, but I don't think we use it anymore.

Reviewed By: quark-zju

Differential Revision: D24284643

fbshipit-source-id: 184697052c0b7000ad9765b4033964b4d382b811
2020-10-14 10:30:32 -07:00
Xavier Deguillard
bc351a5891 templater: decode the result of diff()
Summary:
Template filters are expecting the output of the template to be a unicode
string, not a byte string. Thus we need to ensure that diff return a unicode
string.

Reviewed By: DurhamG

Differential Revision: D24290908

fbshipit-source-id: 70899249c3e26edd1714fd901fa6af405b61e544
2020-10-14 09:00:53 -07:00
Lukas Piatkowski
715351d576 mononoke/integration tests: fix test-clienttelemetry.t by disabling features
Summary:
The logic inside `eden/scm/tests/features.py` script enables certain
features of HG on tests based on the name of the test. Mononoke's integration
tests suit reuses eden/scm's tests suit and as a consequence it triggers the
same `features.py` logic. It was fine until D24201934 (f5988c415c) introduced a feature for
test that is named in the same way as one of the Mononoke tests - the
`test-clienttelemetry.t`.

In order to fix this problem Mononoke will pass a `--nofeatures` flag to the
`eden/scm/run-tests.py` invocations that will turn off the usage of
`features.py`.

Reviewed By: farnz

Differential Revision: D24276294

fbshipit-source-id: eb28ed55a05de3b012e37407603c2370adaaad16
2020-10-13 05:28:27 -07:00
Meyer Jacobs
f9958ca35a taggederror: introduce category and transience metadata and precedence
Summary:
This change introduces two new metadata types, Category and Transience, and a mechanism for Category to provide a default Fault and Transience, which can be overriden by the user.

Also introduces a mechanism for attempting to log exceptions which occur during exception logging, falling back to the previous behavior of just swallowing the exception on failure.

Reviewed By: DurhamG

Differential Revision: D22677565

fbshipit-source-id: 1cf75ca1e2a65964a0ede1f072439378a46bd391
2020-10-12 17:17:34 -07:00
Xavier Deguillard
437031872f fix test-import-t.py take 2
Summary: Whitespaces are needed.

Reviewed By: quark-zju

Differential Revision: D24264452

fbshipit-source-id: a7ecfbe1bba7d14f92d5218f18d55361a3f4319e
2020-10-12 15:43:57 -07:00
Jun Wu
a35f8765c0 lock: print actual lockinfo on error
Summary:
Change

  abort: repository repo: timed out waiting for lock held by <lockinfo ...>

to:

  abort: repository repo: timed out waiting for lock held by process '842210' on host 'hostname'

Reviewed By: singhsrb

Differential Revision: D24214462

fbshipit-source-id: 65056ebb9764651b2f0126061fafdfdefaa4e9c6
2020-10-12 11:22:20 -07:00
Xavier Deguillard
964e0106c9 tests: fix test-import-t.py
Summary: The rev numbers are almost gone, remove them from the test.

Reviewed By: sfilipco

Differential Revision: D24255156

fbshipit-source-id: 5cbc4a71c2d3f773c5b474d1edce84ceceb50bf9
2020-10-12 10:18:45 -07:00
Jun Wu
e4b677c6e1 remotefilelog: respect invalidatelinkrev store requirement
Summary:
With invalidatelinkrev, linkrev stored in revlog shouldn't be used.
This makes some tests pass with segmented changelog.

Reviewed By: singhsrb

Differential Revision: D24201944

fbshipit-source-id: 6473c30266c79aa97a955b1a6c867411cc67de2b
2020-10-09 13:53:18 -07:00
Jun Wu
2f1c54ae27 clone: copy modern store files
Summary:
Copy segmented changelog, metalog and mutation store for local clones.
This mainly affects tests.

Reviewed By: singhsrb

Differential Revision: D24201941

fbshipit-source-id: c60da9e2bf982a6f66004415e45178749157745e
2020-10-09 13:53:18 -07:00
Jun Wu
631034a2d6 gpg: remove revnum from output
Reviewed By: singhsrb

Differential Revision: D24201943

fbshipit-source-id: 6998ff68ba50480fb7feb4be3b35de1ea4a3ab4a
2020-10-09 13:53:17 -07:00
Jun Wu
3700c832f9 rebase: remove revnum from output
Reviewed By: singhsrb

Differential Revision: D24201945

fbshipit-source-id: 57b8fa1f98625ff186fac76009b1600f22e001bb
2020-10-09 13:53:17 -07:00
Jun Wu
38113e78ee summary: remove revnum from output
Reviewed By: singhsrb

Differential Revision: D24201936

fbshipit-source-id: a864fd1030dd8641ead46d434daadae6d4fca244
2020-10-09 13:53:17 -07:00
Durham Goode
43fe23f09d py3: fix revlog path encodings
Summary: This is needed to move our hg servers to python 3.

Reviewed By: quark-zju

Differential Revision: D24204056

fbshipit-source-id: cbaf97893f8f77b535952ac290766f0fd5e14f0c
2020-10-09 13:31:58 -07:00
Stefan Filip
befcdb1672 tests: fix test-casefolding
Summary: Broken after sweeping changes to remove revnums from tests.

Reviewed By: quark-zju

Differential Revision: D24220976

fbshipit-source-id: bd513b056bf16872ce028c835d680aaa87938917
2020-10-09 10:26:15 -07:00
Jun Wu
09c28abd20 graft: remove revnum from output
Reviewed By: singhsrb

Differential Revision: D24201937

fbshipit-source-id: bad22b741410ef990a709f77de5f843de7c3b5dc
2020-10-08 22:33:08 -07:00
Jun Wu
ec02fad2c6 bisect: remove revnum from output
Reviewed By: singhsrb

Differential Revision: D24201938

fbshipit-source-id: a6443465feac1ba43962e44a679091f655b24d16
2020-10-08 22:33:08 -07:00
Jun Wu
b01b79cf76 backout: remove revnum from output
Reviewed By: singhsrb

Differential Revision: D24201935

fbshipit-source-id: e1f4a9c9c82f401cc5b5ed9df464a58da64a0b65
2020-10-08 22:33:07 -07:00
Jun Wu
16aa4415c3 template: remove revnum from children template
Reviewed By: singhsrb

Differential Revision: D24201939

fbshipit-source-id: 573da6fbb1577b5e25566d238dccca33235494cd
2020-10-08 22:33:07 -07:00
Jun Wu
aed62002d1 commit: remove revnum from commitstatus
Summary: Do not show revnum or "changeset" after commit (with --debug or --vrebose).

Reviewed By: singhsrb

Differential Revision: D24201942

fbshipit-source-id: 2f0d15711df67070e50d4bf30f0b1b4401d85524
2020-10-08 22:33:07 -07:00
Jun Wu
f5988c415c tests: enable segmented changelog for supported tests
Summary: Enable segmented changelog for ~400 tests. They pass without changes.

Reviewed By: singhsrb

Differential Revision: D24201934

fbshipit-source-id: 7972c5548fd8b6b6f49b6bb837433d72a6f6cbd9
2020-10-08 22:33:07 -07:00
Jun Wu
140456fc29 bookmarks: remove revnum from non-plain bookmark output
Summary:
Remove revnum from `hg bookmarks` output.
`HGPLAIN=1` output is not changed for compatibility.

Reviewed By: singhsrb

Differential Revision: D24191788

fbshipit-source-id: 3a873b7baba8f1dfe7b3ec690688313ca1134032
2020-10-08 22:33:07 -07:00