Commit Graph

3085 Commits

Author SHA1 Message Date
Jun Wu
eea00a2cb1 dag: add an API for DagAlgorithm identity
Summary:
See the previous diff for context. The new API will be used to check if two
dags are compatible.

Note: It can cause false positive on compatibility checks, which need a
more complex solution. See D25607513 in this stack.

Reviewed By: sfilipco

Differential Revision: D25598079

fbshipit-source-id: f5fc9c03d73b42fadb931038fe2e078881be955f
2020-12-18 16:56:42 -08:00
Jun Wu
3fc8f38541 changelog2: add "lazytext" backend
Summary: The backend is designed to be used by the "debugsegmentclone" command, which does not write revlog.

Reviewed By: sfilipco

Differential Revision: D25624786

fbshipit-source-id: e145128c7b41d78fed495f8da540169f741b674d
2020-12-18 16:47:11 -08:00
Jun Wu
207f755dc0 hgcommits: make revlog optional for the hybrid backend
Summary: This makes it possible to add new commits in a repo without revlog.

Reviewed By: sfilipco

Differential Revision: D25602527

fbshipit-source-id: 56c27a5f00307bcf35efa4517c7664a865c47a43
2020-12-18 16:47:11 -08:00
Jun Wu
78005ae5a5 py3: fix test-run-tests.py
Summary:
When running `python3 run-tests.py test-run-tests.py`, some bytes were printed
with `b` prefix.  Convert them to `str`.

Reviewed By: DurhamG

Differential Revision: D25642164

fbshipit-source-id: f1103b24ad88d0d024f6be546bf632141f06ebd1
2020-12-18 10:38:19 -08:00
Liubov Dmitrieva
18b8c66439 introduce some delay befor writing warning
Summary:
The warning will go to debug level logs if the delay is not reached.

The messages about the locks make profoundly bad effect on attitude to commit cloud even if the delay is just 1 second (that is a reasonable delay).

Reviewed By: quark-zju

Differential Revision: D25587459

fbshipit-source-id: 9a09484d590ba04d17a881e0c9c5d543686b934f
2020-12-18 06:32:00 -08:00
Durham Goode
cfd7ccc828 configs: add Python support for allowed_locations list
Summary:
configs.allowedlocations restricts what configs can be loaded to a
certain set of files. This will enable us to deprecate all old config locations.

This diff adds Python support and a high level test.

Reviewed By: quark-zju

Differential Revision: D25539736

fbshipit-source-id: fa2544379b65672227e0d9cf08dad7016d6bbac8
2020-12-17 06:37:54 -08:00
Durham Goode
dce0faa9b1 configparser: add allowed_location criteria to config verifier
Summary:
We want to start disallowing non-approved config files from being
loaded. To do that, let's update the config verifier to accept an optional list
of allowed locations. If it's provided, we delete any values that came from a
disallowed location.

This will enable us to prune our config sources down to rust configs,
configerator configs, .hg/hgrc, and ~/.hgrc.

Reviewed By: quark-zju

Differential Revision: D25539738

fbshipit-source-id: 0ece1c7038e4a563c92140832edfa726e879e498
2020-12-17 06:37:54 -08:00
Jun Wu
63bb40f9c4 ui: move push/pull messages to stderr
Summary:
Those messages like "pulling from ...", "added n commits ..." belong to stderr.

This makes it possible for us to turn on verbose output for auto pull, without
breaking tools that parses stdout.

Reviewed By: sfilipco

Differential Revision: D25315955

fbshipit-source-id: 933f631610840eb5f603ad817f7560c78b19e4ad
2020-12-16 20:12:04 -08:00
Jun Wu
aef162f7a1 dag: add an API for IdMap identity
Summary:
It turns out `Arc::ptr_eq` is becoming unreliable, which will cause fast paths
to be not used, and extreme slowness in some cases (ex. `public & nodes`
iterating everything in `public`).

This diff adds an API for an IdMap to tell us its identity. That identity is
then used to replace the unreliable `Arc::ptr_eq`.

For an in-memory map, we just assign a unique number (per process) for its
identity on initialization. For an on-disk map, we use the type + path to
represent it.

Note: strictly speaking, this could cause false positives about
"maps are compatible", because two maps initially cloned from each other
can be mutated differently and their map_id do not change. That will
be addressed in upcoming diffs introducing a more complex but precise way to
track compatibility.

Reviewed By: sfilipco

Differential Revision: D25598076

fbshipit-source-id: 98c58f367770adaa14edcad20eeeed37420fbbaa
2020-12-16 20:08:41 -08:00
Jun Wu
c9a701d104 pull: update visibility efficiently via repo.pull API
Summary:
With selectivepull we can tell visibility directly about which heads to add,
instead of adding all nodes in a changegroup.

Note: This does not affect the pull command, nor exclude public commits yet.

Reviewed By: markbt

Differential Revision: D25562090

fbshipit-source-id: aa5f346f33058dfdb3b2f23f175e35b5d3c30a1d
2020-12-16 17:12:55 -08:00
Jun Wu
35094bad4e remotenames: move a few methods to core
Summary: They will be used in core later.

Reviewed By: markbt

Differential Revision: D25562093

fbshipit-source-id: 4402a629a09920fd4c6f85cb8e777446bb218a37
2020-12-16 17:12:55 -08:00
Liubov Dmitrieva
d10229e533 add log path for the background backups to hg cloud status output
Summary:
It always takes a bit of time to find the logs. Since we do have scm daemon onces in the output of `hg cloud status`,
it would be nice to have the onces from background backup as well.

Reviewed By: markbt

Differential Revision: D25560145

fbshipit-source-id: cdf5d76c7c3ebb1492559d32935f9301452a1cd5
2020-12-16 06:23:00 -08:00
Liubov Dmitrieva
7076cda874 fix order of arguments in error message
Summary: The order has been incorrect and led to a confusing message

Reviewed By: krallin

Differential Revision: D25559963

fbshipit-source-id: 4fcb3e53cedcb08675b60b25cbb5da2ca52c08ed
2020-12-16 06:23:00 -08:00
generatedunixname89002005307016
1e1a7fa10d suppress errors in eden - batch 1
Differential Revision: D25562362

fbshipit-source-id: ec62c64396f61335f6cfe8f355ba977bfd1da031
2020-12-15 15:22:22 -08:00
Jun Wu
7c1d264b58 py3: fix debugshell compatibility without chg / with demandimport
Summary:
IPython is incompatible with Python 3 demandimport. Disable demandimport to
make it work.

Reviewed By: singhsrb

Differential Revision: D25542394

fbshipit-source-id: 293880dff62e98895bc1ae2d3328d4af25b8218f
2020-12-15 14:52:26 -08:00
Jun Wu
ec0b533381 ui: make ui.debug write to stderr
Summary:
Debug output belongs to stderr.

This makes it possible to turn on debug output without breaking programs
parsing stdout.

Reviewed By: singhsrb

Differential Revision: D25315954

fbshipit-source-id: c7813a824fbf6640cb5b80b5ed2d947e7059d53e
2020-12-15 12:07:47 -08:00
Jun Wu
d7f7bc0181 smartlog: always show "."
Summary:
With `collapse-obsolete`, `.` can be obsoleted and in the middle of a stack and
not shown up. That can be confusing. Make the smartlog revset always show the
`heads` passed in. If `.` is in `heads` (the default), then show it.

Reviewed By: DurhamG

Differential Revision: D24696595

fbshipit-source-id: 7deab109d0e0ae5e703928252bc63312d936955f
2020-12-15 11:52:02 -08:00
Robert Balicki
380cec1a54 Suggest connecting to the VPN if hg update doesn't work
Summary: * suggest connecting to the VPN if hg update doesn't work

Reviewed By: sfilipco

Differential Revision: D25551017

fbshipit-source-id: 575f29cce4ab2719f2faae86616fdd9aac739f5f
2020-12-15 08:50:38 -08:00
Thomas Orozco
30f6b1de5d infinitepush: submit LFS blobs via remotefilelog
Summary:
If Rust LFS is in use, we currently don't upload LFS blobs to commit cloud.
This is problematic because if you're going to Mononoke that means you can't
upload, and if you're going to Mercurial that means you're silently not backing
up data.

Reviewed By: StanislavGlebik

Differential Revision: D25537672

fbshipit-source-id: fd61f5a69450c97a0bc0895193f67fd22c9773fb
2020-12-15 08:20:29 -08:00
Durham Goode
d04c074d89 configs: fix issue with config removal
Summary:
The config verifier would remove items from the values list if they
were disallowed. To do this, it iterated through the values list backwards,
removing bad items.  In some cases it stored the index of a bad value for later
use, but because it was iterating backwards and removing things, the indexed it
stored might not be correct by the time the loop is done. To fix this, let's go
back to iterating forwards.

Reviewed By: quark-zju

Differential Revision: D25539737

fbshipit-source-id: 87663f3c162c690f3961b8075814f3467916cb4b
2020-12-15 08:03:50 -08:00
Jun Wu
cb6bd19903 streams: restore commit text prefetching behavior
Summary:
Restore the behavior disabled by D25350916 (49c6f86325). This time it no longer runs Python
logic in background threads.

Reviewed By: sfilipco

Differential Revision: D25513054

fbshipit-source-id: 0220ccb37e658518d105bba04f45424c9fcfe142
2020-12-14 19:10:44 -08:00
Jun Wu
3a371e7aa7 hgcommits: be aware of hardcoded commit hashes in more places
Summary:
Make `get_commit_raw_text` aware of hg's hardcoded commit hashes: NULL_ID and
WDIR_ID. Previously, only `stream_commit_raw_text` is aware of it.

This makes it a bit more compatible when used in more places.

Reviewed By: sfilipco

Differential Revision: D25515006

fbshipit-source-id: 08708734a28f43acf662494df69694988a5b9ca0
2020-12-14 19:10:44 -08:00
Jun Wu
843ef6aab2 pydag: add API to get commit text in batch
Summary:
Unlike streamcommitrawtext, the new API does not put Python logic to a
background thread. This will make it easier to reason about Python logic as
they do not need to be thread-safe, and we don't need to think about Python GIL
deadlocks in the Rust async world.

Reviewed By: sfilipco

Differential Revision: D25513057

fbshipit-source-id: 4b30d7bab27070badd205ac1a9d54bae7f1f8cec
2020-12-14 19:10:44 -08:00
Jun Wu
1c2135dd15 hgcommits: add fallback 1-by-1 fetching for the hybrid backend
Summary:
Previously, only the batch fetching, or the stream fetching APIs will
actually fetch commit remotely. The 1-commit fetching API does not have
the network side effect, with the hope that we can migrate all usecases
to stream or batch fetching.

Practically it's quite difficult to migrate all use-cases, and the Python
layer has to have a fallback 1-by-1 fetching. Now let's just move that
fallback to Rust to simplify the code. The fallback in the Rust code
is by the default impl of get_commit_raw_text.

Reviewed By: sfilipco

Differential Revision: D25513056

fbshipit-source-id: b3c615397d33b8d35876dc23ca7b95173783ef80
2020-12-14 19:10:44 -08:00
Jun Wu
8d66ade1de hgcommits: add API to get commit text in batch
Summary: The API will be used in Python bindings to avoid running Python in background threads.

Reviewed By: sfilipco

Differential Revision: D25513055

fbshipit-source-id: a108b55115271a256c0d43e0ff7b82c0b209be81
2020-12-14 19:10:44 -08:00
Jun Wu
1a5755de2c smartset: make __iter__ do prefetch
Summary:
Previously only `iterctx` does prefetch. Make `__iter__` do prefetch via `iterctx`.
The old `__iter__` without prefetching was renamed to `iterrev`.

Reviewed By: sfilipco

Differential Revision: D24365404

fbshipit-source-id: db5c687066794257719bb64c673dc384b5460ff1
2020-12-14 13:12:43 -08:00
Jun Wu
4cde87b34b smartset: drop repo from iterctx
Summary:
Now smartset has a reference to repo. It does not need `repo` from external
source.

Reviewed By: sfilipco

Differential Revision: D24365405

fbshipit-source-id: 8a43697b7b84a8a41691ed8f095c271107a90f16
2020-12-14 13:12:42 -08:00
Jun Wu
1a1b3a3cb5 smartset: make repo required for baseset
Reviewed By: sfilipco

Differential Revision: D24365401

fbshipit-source-id: 4d0ee6d27717c1aa966086af68492295aa6ed372
2020-12-14 13:12:42 -08:00
Jun Wu
99c2440f9e smartset: make repo required for nameset
Reviewed By: sfilipco

Differential Revision: D24365402

fbshipit-source-id: 72282634dfba862eab39cef176f0073350921ffc
2020-12-14 13:12:42 -08:00
Jun Wu
0ef3c1aa32 smartset: make repo required for generatorset
Reviewed By: sfilipco

Differential Revision: D24365403

fbshipit-source-id: 8cdee2d344b6af02804c6c0dca3243057df0a643
2020-12-14 13:12:42 -08:00
Jun Wu
01553ffe03 smartset: make repo required for idset
Reviewed By: DurhamG

Differential Revision: D24365406

fbshipit-source-id: 17cbe8e5fc1ea6025b5006737dedb6f744c009a4
2020-12-14 13:12:41 -08:00
Jun Wu
1327b6ec3b smartset: add optional repo reference to smartset objects
Summary:
This will make `__iter__` to be able to do proper prefetch, or make it possible
for `__iter__` to return node instead of rev. To avoid cycles, weakref is used.

The smartset types are used widely. It's hard to migrate all callsites at once.
For now, `repo` is optional. Later, it will be required.

Reviewed By: DurhamG

Differential Revision: D24365400

fbshipit-source-id: 5dd40e3d930893c39f16da8f3169b026c8933bd2
2020-12-14 13:12:41 -08:00
Jun Wu
d55eae5fdb tests: remove test-extension.t
Summary:
The test is one of the most fragile tests (ex. it has traceback that is
fragile).  Since we don't support 3rd party extensions the test is less
meaningful. Let's remove it.

Reviewed By: singhsrb

Differential Revision: D25514727

fbshipit-source-id: cc3491597ba38ad84c4c2b69a275da6505ae2207
2020-12-14 13:12:41 -08:00
Jun Wu
90f40e79bd tests: mark test-atexit-epipe.t as chg incompatible
Summary:
It has a high chance of failing with:

```
   $ hg --config extensions.b=b.py --config 'pager.pager=head -1' init foo2
+  close failed in file object destructor:
+  IOError: [Errno 9] Bad file descriptor
   line1
```

when running with chg together with other tests.

Reviewed By: singhsrb

Differential Revision: D25514728

fbshipit-source-id: d9d2adea97bc6a436341be6f6d9aac43cb78c90a
2020-12-14 13:12:41 -08:00
Jun Wu
abc69ba70a remotefilelog: prefetch commit text during adjustlinknode
Summary: `adjustlinknode` needs commit text to resolve manifest. Prefetch it.

Reviewed By: DurhamG

Differential Revision: D24324798

fbshipit-source-id: 160f05947f3b54962acfa21ed6c973e196273e74
2020-12-14 13:12:41 -08:00
Jun Wu
fa8817c9b4 revset: prefetch text for ancestorsaged()
Summary:
This is a bit tricky - look-ahead does not work because the content of commit
texts decide whether the commit is in the resulting set or not.

Add a special case for `ancestorsaged` so it does not end up with 1-by-1
fetching.

Reviewed By: sfilipco

Differential Revision: D24324797

fbshipit-source-id: 844ac6a6637e25900eea00d99704a59dfc7a0345
2020-12-14 13:12:40 -08:00
Jun Wu
af45384c96 revset: prefetch text for related revset functions
Summary:
Some revset function need commit text data to be efficient. Prefetch commit
text explicitly for them.

Reviewed By: DurhamG

Differential Revision: D24322227

fbshipit-source-id: 49c0b4f8cd9d9a744e10d169a1b1247faa8edd8e
2020-12-14 13:12:40 -08:00
Kostia Balytskyi
2db894e581 wireproto: fix incorrect name in timefunction
Reviewed By: krallin

Differential Revision: D25531699

fbshipit-source-id: ed620a7f00351251d3670ed1ce43240a584e76cd
2020-12-14 07:57:46 -08:00
Durham Goode
2333b25e75 tests: fix rust store test on windows
Summary: Looks like the permissions are different there. Let's glob it out.

Reviewed By: singhsrb

Differential Revision: D25507359

fbshipit-source-id: 6a5c19e41879798b829d9b6e79eba3009249c20c
2020-12-14 06:53:24 -08:00
generatedunixname89002005325677
9c5eaa7c81 Daily arc lint --take RUSTFMT
Reviewed By: zertosh

Differential Revision: D25527687

fbshipit-source-id: 25c98dcb607ec7f469099b68f432f3412ca9988e
2020-12-14 03:56:02 -08:00
Stanislau Hlebik
cbe07f0d17 infinitepush: select infinitepush path only for scratch bookmarks
Summary:
At the moment "hg pull -B bookmark" always fetches from infinitepush path even
if we do something like "hg pull -B master".

Let's fetch from infinitepush only if a bookmark matches scratch mather

Reviewed By: markbt

Differential Revision: D25460577

fbshipit-source-id: 6563dcd3423c6a7a70ea1c1f7acdaf5db5e21875
2020-12-13 23:48:22 -08:00
Arun Kulshreshtha
f98273063a auth: make auth_for_url validate certs
Summary: Make the auth crate validate the user's certificate before returning it. This way we can catch invalid certs before trying to use them.

Reviewed By: sfilipco

Differential Revision: D25454687

fbshipit-source-id: ad253fb433310570c20f33dbd0d0bf11df21e966
2020-12-11 18:55:03 -08:00
Arun Kulshreshtha
a8a9fa3246 auth: add x509 module for validating certs
Summary: Add a new module that can parse X.509 certificates and detect common issues (e.g., the certificate is missing, corrupt, or expired). This should allow us to provide better UX around certificate errors.

Reviewed By: sfilipco

Differential Revision: D25440548

fbshipit-source-id: b7785fd17fa85f812fd38de09e79420f4e256065
2020-12-11 18:55:03 -08:00
Jun Wu
ec7c0659c9 dag: relax trait bounds for AbstractNameDag
Summary: This makes it more flexible.

Reviewed By: kulshrax

Differential Revision: D24467604

fbshipit-source-id: 63023cf0dde2fb7eac592ac79008e4b7a62340c1
2020-12-11 17:03:37 -08:00
Xavier Deguillard
49de273c77 revisionstore: handle redacted blob in LFS
Summary:
When a blob is redacted server side, the http code 410 is returned.
Unfortunately, that HTTP code wasn't supported and caused Mercurial to crash.
To fix this, we just need to store a placeholder when receiving this HTTP code
and simply return it up the stack when reading it.

Reviewed By: DurhamG

Differential Revision: D25433001

fbshipit-source-id: 66ec365fa2643bcf29e38b114ae1fc92aeaf1a7b
2020-12-11 12:28:38 -08:00
Durham Goode
d7af02abb1 revisionstore: fix paths for local indexedlog
Summary:
There was a bug with local-data indexedlog storage where it
wasn't applying the appropriate suffix, so tree data was being stored in
.hg/store/indexedlogdatastore just like file data. Let's fix that and add a
test.

Reviewed By: quark-zju

Differential Revision: D25469917

fbshipit-source-id: 731252f924f9a8014867fc077a7ef10ac9870170
2020-12-11 06:58:27 -08:00
Wez Furlong
62f122c13a edenscm: fsmonitor: sort blackbox shortlists
Summary:
The integration tests rely on the ordering, but watchman doesn't
define an ordering in its file results, so the current implementation is
implicitly tied to the iteration order of the InMemoryView indices inside
watchman.

In D25414061 that implementation is changed to be derived from the order
in which entries are added to a change journal, making the results more
likely to match the readdir directory iteration order.

Since the blackbox fsmonitor integration tests have a defined ordering in their
output, we need to sort these results here.

Reviewed By: singhsrb

Differential Revision: D25468807

fbshipit-source-id: aa83d82cca0ecdbfd1d4bb1309b9f3c31f6c3136
2020-12-10 18:48:37 -08:00
Jun Wu
dcf4957619 dag: make parent function async
Summary: Make the parent function used by various graph building functions async.

Reviewed By: sfilipco

Differential Revision: D25353612

fbshipit-source-id: 31f173dc82f0cce6022cc2caae78369fdc821c8f
2020-12-10 12:37:36 -08:00
Jun Wu
bffffb2415 dag: remove IdMapBuildParents
Summary:
It is no longer needed for building segments (replaced by "prepared flat
segments"). Remove it.

Reviewed By: sfilipco

Differential Revision: D25353613

fbshipit-source-id: aede9e33c3217a61b5b14aae5b128d8953bc578e
2020-12-10 12:37:36 -08:00
Jun Wu
ad6f25addc dag: make IdConvert async
Summary: Make IdConvert async and migrate all its users.

Reviewed By: sfilipco

Differential Revision: D25350915

fbshipit-source-id: f05c89a43418f1180bf0ffa573ae2cdb87162c76
2020-12-10 12:37:35 -08:00