Commit Graph

44650 Commits

Author SHA1 Message Date
Kostia Balytskyi
3e70d22bf6 hgbuild: run all tests with hg.rust rather than with python-based hg
Summary: Since we are using `hg.rust` with everything, we should also test it.

Differential Revision: D13121588

fbshipit-source-id: 3a4e07c1d70c279f008812d026bf9a7c74aa3b2e
2018-11-22 07:41:41 -08:00
Liubov Dmitrieva
678562e8cd commitcloud: improve TLS error messaging
Summary: in case of VIP TLS errors comes as socket errors.

Reviewed By: DurhamG

Differential Revision: D13159635

fbshipit-source-id: 80204718ea117e3694199a8686a151ff8aae2634
2018-11-21 13:23:41 -08:00
Durham Goode
39d85694ed pushrebase: don't prepopulate manifest caches
Summary:
This had a nasty bug where it manually computed the cached base text
using the full manifest text. When operating in a treeonly repository, this
resulted in corrupt deltas. The only reason we didn't hit it earlier is because
the secondary stackpush path was added and doesn't have this issue. But it
couldn't handle merge commits, which is why we hit this when it came time to add
a merge commit.

Now that the main repo that needed this optimization is treeonly, we don't need
this precaching anymore.

Reviewed By: mitrandir77

Differential Revision: D13154342

fbshipit-source-id: 5cb74167665a60a36e4a6d926f3f9f1c5c7bbef1
2018-11-21 08:33:48 -08:00
Durham Goode
9a40d36329 pushrebase: add test demonstrating corruption with treeonly
Summary:
There's a corruption that occurs because pushrebase caches a fulltext
instead of a tree. Let's write a test to expose it. The next diff will fix it.

Reviewed By: mitrandir77

Differential Revision: D13154341

fbshipit-source-id: e2ac040c59c677b414b6eddd39d7dc2ebb25f082
2018-11-21 08:33:48 -08:00
Kostia Balytskyi
11336ed4d5 tests: fix grep finding grep in rage output
Summary:
Fix tests.
This makes sure that the grep process itself is not matched.

Reviewed By: farnz

Differential Revision: D13138516

fbshipit-source-id: d29829584c6fe34dd60345b2812682d4cb9d51e4
2018-11-21 07:30:31 -08:00
Kostia Balytskyi
e6586e0ab8 lz4revlog: make it side-effect free at import time
Summary:
Mercurial extensions are expected to be side-effect free at import time.

Move setup code to `uisetup` so they only get executed if the extension is enabled.

This helps chg's case where `lz4revlog` is always pre-imported.

Reviewed By: quark-zju

Differential Revision: D13118788

fbshipit-source-id: a7801057c01b6918bb4902a326b92f1c17f2707a
2018-11-21 06:31:03 -08:00
Liubov Dmitrieva
665ea1b9ee commitcloud: print message in hg sl --all when commits are omitted
Summary:
When we have omitted syncing commits from the cloud workspace, give the user
advice on what to do to fetch them if they run `hg sl --all`.

Reviewed By: markbt

Differential Revision: D13085505

fbshipit-source-id: 624ce6cbf4cb2194ab5ffbc09c1ac3e073932249
2018-11-21 06:01:46 -08:00
Kostia Balytskyi
c646d8aa2a hg.rust: canonicalize the main binary address
Summary:
This is important on OSX where `current_exe` will return the symlink address if `hg.rust` is a symlink.
Therefore, if you create a symlink to the `hg.rust` in the repo (like tests do), repo Python code won't be picked up, and the system code will be.

Reviewed By: mitrandir77

Differential Revision: D13138333

fbshipit-source-id: ffdf27329609d77bee4b8a2eecc47e02cb2dd5c8
2018-11-21 05:47:45 -08:00
Chad Austin
e62d26feae don't swallow Python errors in tree iteration
Summary:
During the sequence of events that led up to S169085 we tried an hg
push that resulted in an error that looked like this: "RuntimeError:
std::exception". I believe that error was caused by an uncaught
pyexception, replacing Python's current thread exception with a
useless one.

Catch the pyexception and deal with it appropriately.

Reviewed By: quark-zju

Differential Revision: D13112735

fbshipit-source-id: 301d899543ae95084b890c19b00322e69ded07b2
2018-11-20 15:00:43 -08:00
Mark Thomas
38cc6a356c help: fix stablerev help summary in test
Differential Revision: D13118834

fbshipit-source-id: 4cbcdf261e145acdfc260f2884d39ca0420b606f
2018-11-19 06:38:28 -08:00
Kostia Balytskyi
e788232917 chg: make sure CHGDISABLE is set for non-chg tests
Summary: Now that `chg` is embedded into `hg.rust`, we need to explicitly disable it while tests run.

Reviewed By: quark-zju

Differential Revision: D13021560

fbshipit-source-id: 843045f3bc9d9866e421f0bf0aa132849e5b338f
2018-11-19 06:15:32 -08:00
Kostia Balytskyi
9bb35f3233 chg: tell original cmd to not restart chg
Summary:
When `chg` decides, that the command is unsupported, it executes the real hg
instead. Now that the real `hg` and the `chg` can be the same binary, we need
to make sure that `chg` is definitely not re-executed. Passing `CHGDISABLE`
achieves that.

Reviewed By: markbt

Differential Revision: D13118766

fbshipit-source-id: e41a2c230d17c4c26e590d648e4d0b722cf41ee5
2018-11-19 06:15:32 -08:00
Mark Thomas
a98601aa87 commitcloud: release lock when testing if it's taken
Summary:
In backuplockcheck we try to take the lock to see if it's currently in use.  If
it isn't, then we end up taking the lock, but don't release it.  We should
release it.

Reviewed By: liubov-dmitrieva

Differential Revision: D13085506

fbshipit-source-id: 4b38ec11385d754d07ad2d15ff14b43d37325263
2018-11-19 03:18:28 -08:00
Durham Goode
6040777902 hgsubversion: improve performance with treemanifests
Summary:
hgsubversion has some logic that tries to read the list of files in a
given directory. In a tree-only world, it turns the entire tree into a text
file, then splits it by line and bisects over it.

Let's instead use the walk function, which is optimized for this kind of thing
on trees.

Reviewed By: quark-zju

Differential Revision: D12969497

fbshipit-source-id: ef3af9e0022978d6a4922cbb464bfd14248f5501
2018-11-16 11:06:09 -08:00
Kostia Balytskyi
a07b4a97bc hg.rust: ensure we rebuild if chg changes
Summary: For the sake of incremental builds and `make local`.

Reviewed By: quark-zju

Differential Revision: D13058780

fbshipit-source-id: f59dc235e75232b475ca0c2504b47ff0d4a64c8a
2018-11-16 02:35:09 -08:00
Jun Wu
61f0a3da45 tests: add a test-check test that runs fix-code.py
Summary:
Add "--dry-run" for fix-code.py and use it in test-check.
This avoids license header and version = "*" issues.

Reviewed By: ikostia

Differential Revision: D10213070

fbshipit-source-id: 9fdd49ead3dfcecf292d5f42c028f20e5dde65d3
2018-11-15 18:54:06 -08:00
Jun Wu
616306543b codemod: use explicit versions in Cargo.toml
Summary:
This is done by running `fix-code.py`. Note that those strings are
semvers so they do not pin down the exact version. An API-compatiable upgrade
is still possible.

Reviewed By: ikostia

Differential Revision: D10213073

fbshipit-source-id: 82f90766fb7e02cdeb6615ae3cb7212d928ed48d
2018-11-15 18:54:06 -08:00
Jun Wu
47dfaa269f fix-code: add logic to fix Cargo.toml
Summary:
`version = "*"`  is not a good pattern. This script reads Cargo.lock and writes
down the version explicitly. This helps the OSS build.

Reviewed By: ikostia

Differential Revision: D10213074

fbshipit-source-id: 39720d895a5788b68d3e9d183d5e8941abc8c048
2018-11-15 18:54:06 -08:00
Jun Wu
65a5c67b42 doc: rewrite WritingExtensions
Summary:
This is a shorter, rewritten version of "WritingExtensions".

"WritingExtensions" is too verbose and some code examples are outdated.

Reviewed By: markbt

Differential Revision: D10287050

fbshipit-source-id: f989172c4a683142983e17d3fa9dbddfd8252586
2018-11-15 18:22:31 -08:00
Mark Thomas
a3c411c1e8 commitcloud: omit older commits when pulling during sync
Summary:
Add a new config option: `commitcloud.max_sync_age`.  When set, commit cloud
will not pull in any commits that are older than this when it is joining or
syncing.  The commits are still nominally in the cloud workspace, we just
save join or sync time by not including the commits.

Reviewed By: liubov-dmitrieva

Differential Revision: D13062470

fbshipit-source-id: 17a4bdb4095766a83a4bf6d4151ae86b39edf59c
2018-11-15 12:16:19 -08:00
Marla Azriel
2889784e08 commands: include logopts for smartlog
Summary: This commit reverts an earlier change in D12950765 that removed options from 'hg smartlog'

Reviewed By: kulshrax

Differential Revision: D13029915

fbshipit-source-id: f514dca841cf9a48a46255c4eb0b376d8f0d2761
2018-11-14 16:24:01 -08:00
Xinyuan Wang
c76c75a938 lfs: log upload size and time
Summary: Log the uploading time spend and the downloading size when hg is using lfs

Reviewed By: ikostia

Differential Revision: D12993962

fbshipit-source-id: c53b189a12c60eece47dbbab0852fcfea9471363
2018-11-14 11:05:07 -08:00
Phil Cohen
5564b837bc rebase: always return 0 on noop rebase
Summary:
tweakdefaults used to do this if configured.

After dicussion with the team, we decided that returning 1 was not a helpful design decision, so we're going to fold this in here and everywhere.

Reviewed By: singhsrb

Differential Revision: D13050548

fbshipit-source-id: 66e834ea503e4b1339e369495a9729b951024a6d
2018-11-13 16:35:31 -08:00
Durham Goode
f1e2bc6543 tests: fix gendoc warning
Summary: This broke the tests

Reviewed By: quark-zju

Differential Revision: D13047589

fbshipit-source-id: a2ee769a64f4a07042d852b7c74e67b0f3363f8b
2018-11-13 12:13:33 -08:00
Lukas Piatkowski
7f88618721 hg rage: add environment variables to the output
Reviewed By: phillco

Differential Revision: D13042931

fbshipit-source-id: bbd283021bce67f42811664db49e135e5564bb5d
2018-11-13 10:14:13 -08:00
Liubov Dmitrieva
d2328e5e66 commit cloud: add workspace option for hg sl and hg ssl commands
Summary:
These commands should be able to use the full workspace name as well
Also the fix for commit cloud test file is included

Reviewed By: markbt

Differential Revision: D13042297

fbshipit-source-id: ac8c907292e2d15b72f56ef1cc831add5523b990
2018-11-13 08:08:28 -08:00
Anastasiya Zhyrkevich
baad48586f lfs debug command to receive a bunch of objects
Summary:
debugcommand for receiving several lfs objects from server
Analog to the funtion `debuglfsreceive`, but for several objects

Will be used in replay traffic for Mononoke LFS.

Requires URL and an iterable with oids and sizes.
Format required:
url, (oid1, size1, oid2, size2, ... )

Write all files to the console.

Reviewed By: quark-zju

Differential Revision: D12956274

fbshipit-source-id: f83dd0636b2ad197cace9633222d0f1ed8191dab
2018-11-13 03:11:18 -08:00
Phil Cohen
4b9fdcd042 stablerev: add getstablerev() revset
Summary:
This allows us to expose `arc pull`'s functionality directly from hg.

There are three parts:

- Create a revset that runs a program, reads the stdout and looks up that commit if it exists
- Auto-pulling if the commit doesn't exist
- Supporting an optional argument (target) that's passed to the script

Reviewed By: DurhamG

Differential Revision: D10524541

fbshipit-source-id: 7493c5592e272f9e8a87f109cec1426d44935ecc
2018-11-12 13:38:49 -08:00
Pavel Aslanov
41c2048ce6 record updated onto rev
Summary:
We need to record updated onto rev instead of guessing it in pushrebase replayer.

I will not land this diff until mononoke db schema is update D12923144, D12997125

Reviewed By: quark-zju

Differential Revision: D12922833

fbshipit-source-id: 11c6411c392ca9092be53ffba8baa074faf3a996
2018-11-12 04:17:37 -08:00
Durham Goode
b6a2776924 remotefilelog: make pending local pack writes available for reads
Summary:
Previously, if we were writing local data to pack files we weren't able
to read that data until the pack file had been flushed. Let's add those mutable
packs to the union store so we can see there data.

This is important for unblocking use of hggit and hgsubversion since they may
import many dependent commits in a single transaction.

Reviewed By: quark-zju

Differential Revision: D12959497

fbshipit-source-id: 405c0c5c1e8fc84bc8ffef827a84e91d57eb95d8
2018-11-11 21:37:52 -08:00
Durham Goode
f1d6212c00 remotefilelog: add test showing issue with packlocaldata
Summary:
The config to write local packs has an issue with reading data from
currently pending mutable packs. Let's add a test demonstrating this, and in the
next diff we'll fix it.

Reviewed By: quark-zju

Differential Revision: D12959501

fbshipit-source-id: 5bcb278ccb977cea83e8cd594a404816d4f8f1fa
2018-11-11 21:37:52 -08:00
Durham Goode
729133f6cf remotefilelog: move mutable packs onto fileslog
Summary:
Previously these were stored on the remotefilelogcontentstore, which is
a weird place since it's generally responsible for loose files. Now that we have
the fileslog abstraction let's move the mutable packs on to it.

This mirrors the manifestlog pattern.

Reviewed By: quark-zju

Differential Revision: D12959496

fbshipit-source-id: 25649570a44b50e9baa558b85ba00605883fd403
2018-11-11 21:37:52 -08:00
Durham Goode
ee9beb6c2f remotefilelog: move mutablemanifeststore into shallowutil
Summary:
A future diff will use the same mutable*store pattern to allow pending
file mutable packs to be read from the store. The mutablemanifeststore is
generic and can be reused, so let's move it and rename it.

Reviewed By: quark-zju

Differential Revision: D12959493

fbshipit-source-id: 82710b4d157eb3194440ea630dd458b382896a39
2018-11-11 21:37:52 -08:00
Durham Goode
e9e9022cfe remotefilelog: move file stores onto fileslog
Summary:
Now that all the file store creation logic is inside the fileslog
abstraction, let's also move the storage of all the stores onto this instance,
instead of having them hanging off the repo object.

This matches the manifestlog pattern and gives us better control over the
lifetime of all the filelog stores.

Reviewed By: quark-zju

Differential Revision: D12959500

fbshipit-source-id: f07f8b52bb83a7837e6dc02664bec6111df7a421
2018-11-11 21:37:52 -08:00
Durham Goode
e8a53816b1 remotefilelog: move creation of file stores into fileslog
Summary:
As part of unifying file storage into fileslog, let's move the store
creation logic into fileslog. It still puts the stores on the repo directly, but
a future diff will come back and fix that so the stores are kept on the fileslog
object.

Reviewed By: quark-zju

Differential Revision: D12959498

fbshipit-source-id: f3defc88b34c74c95bf1604f194b1b5883bad24d
2018-11-11 21:37:52 -08:00
Durham Goode
60c80f4f48 remotefilelog: move remotefilelog specific fileslog logic to remotefilelog
Summary:
We had remotefilelog specific logic in core mercurial code. Now that
we have the fileslog abstraction we can hide remotefilelog specific invalidation
in a remotefilelog specific fileslog implementation.

Reviewed By: quark-zju

Differential Revision: D12959495

fbshipit-source-id: ccf224bf9799eb1af74f0dff6021fcbc2eb20d68
2018-11-11 21:37:52 -08:00
Durham Goode
55010e4e41 remotefilelog: introduce the concepts of a fileslog
Summary:
In upcoming diffs I want to introduce lifetime management for
structures receiving writes for file content. We need to be able commit these at
the end of a transaction, and roll them back in the event of an abort.

We already have this, but it's adhoc and inconsistent. Let's introduce the
concept of a fileslog on the repository that is responsible for all filelog read
writes. At the moment it doesn't manage reads, and only governs remotefilelog
writes, but we can extend this later.

This pattern of a top level repo.*log property that manages the reading/writing
of a type of object is already used in the manifestlog and has proven to be
relatively clean.

Reviewed By: quark-zju

Differential Revision: D12959494

fbshipit-source-id: 676aa86c313cb7e48512091a9c19b9452e8f114a
2018-11-11 21:37:52 -08:00
Durham Goode
23f7e93958 remotefilelog: remove writestore concept from union stores
Summary:
Previously, remotefilelog tried to hide which store should receive
writes behind the union store abstraction. This is starting to make things a
little complicated though. For instance, it means that right now both local data
and local history are written via one api on the remotefilelogcontentstore, even
though the remotefilelogcontentstore should only be about data. It also means
that remotefilelogcontentstore had to become aware of pack files, which it can't
read, so it no longer upholds the guarantee that anything written to it can be
immediately read. Overall it's just confusing.

This diff rips out the writestore concept, and instead has the store setup logic
store the writable stores directly on the repo, and changes the write path to
write to those stores directly. Thus removing the notion of pack files from
remotefilelogcontentstore.

A future diff will clean this up even further, and fix the bug where you can't
read data that was just written to a local pack.

Reviewed By: quark-zju

Differential Revision: D12959502

fbshipit-source-id: 85c39c0696febd0972a21f22f3640fd6954901c1
2018-11-11 21:37:52 -08:00
Jun Wu
647f7dfb8e indexedlog: fix misc benchmark
Summary:
The "misc" benchmark requires the base16 module to be public. It was made
private in a previous change. Let's make it public again so the benchmark can
run.

Reviewed By: singhsrb

Differential Revision: D13015031

fbshipit-source-id: 0dc1542803aae290de26651e367898eebfc95e83
2018-11-09 20:49:56 -08:00
Phil Cohen
1d0f1cd7bf hgsubversion: log conflicted paths on failed SVN push
Summary:
This will help us in identifying commonly conflicted files/artifacts.

Alas, I think SVN only includes one path in the error message. But better than nothing.

Reviewed By: quark-zju

Differential Revision: D13009975

fbshipit-source-id: 220bcaa679222718c58e42174f28fd0bbeb618d2
2018-11-09 16:56:35 -08:00
Jun Wu
ef5f621baa progress: drop config to specify condtype
Summary:
The Rust condition implemenation seems working okay. Therefore drop the support
to switch to buggy `threading.Condition`.

Reviewed By: phillco

Differential Revision: D12875128

fbshipit-source-id: 15f41bdc3fceeaa7de7d8cdbd76ee6d70951c107
2018-11-09 15:27:52 -08:00
Marla Azriel
4027f3a7e9 commands: update help text for smartlog, show, diff
Summary: Provides updated help text for hg smartlog, hg show, and hg diff

Reviewed By: markbt

Differential Revision: D12950765

fbshipit-source-id: d442079499abada1740760c8b6d0c27bf4a2b2d3
2018-11-09 13:14:20 -08:00
Liubov Dmitrieva
372c5594b4 Scm Daemon: support cat tokens via passing token type to the URL
Summary: This is the final step to make CAT authentification work

Reviewed By: markbt

Differential Revision: D12975214

fbshipit-source-id: e445ca502f8abaac914140f3f30476d50b3c2fbc
2018-11-09 11:48:50 -08:00
Liubov Dmitrieva
299ecd354a Scm Daemon: support CAT tokens
Summary:
To solve friction with OAuth tokens we will support CAT tokens as well in Scm Daemon.

Icebreaker support has been done in D12942971

CATs tokens can be generated on dev servers without user (via the tool based on TLS certs).

So we are going to use them in the next diff.

This will allow us to enable token-less cloud sync for everyone, scm daemon will use CATs.

Reviewed By: markbt

Differential Revision: D12962342

fbshipit-source-id: 173301387ee446622bf77b2d6bed6934b5ced2c3
2018-11-09 09:51:26 -08:00
Liubov Dmitrieva
8a024ed563 Scm Daemon: make scm daemon more robust with expiration of tokens
Summary:
Basically if Unauthorized it will try to access the token again and restart all the subscriptions
rather than trying to reconnect with the same token in infinite loop.

We know OAuth tokens have potential to be invalidated.

CAT token (that we are going to support as well) will always be valid for some time - like 1 day, so we need a smooth way to recover from  Unauthorized and issue a fresh token.

Reviewed By: markbt

Differential Revision: D12960843

fbshipit-source-id: 630c446c490b0724df38c61507ee555dc7ed7241
2018-11-09 09:51:26 -08:00
Aida Getoeva
1c7a7e8eb2 debugcrdump: add --nobinary flag to avoid dumping binaries
Summary: Jellyfish uses `hg debugcrdump` to get commit data, including binary files. Now we want in jf to be able avoid dumping binary files, so I added `--nobinary` option to crdump

Reviewed By: quark-zju

Differential Revision: D12981968

fbshipit-source-id: 669ef4cf8f6225b911f5085c1a47fb7242f259bf
2018-11-09 08:50:37 -08:00
Jun Wu
6a01a5de06 linelog: update README
Summary: This is a backport of my upstream patch https://phab.mercurial-scm.org/D4147

Differential Revision: D12970974

fbshipit-source-id: ed9d8db2e32818e6e5ab3f23f5a0097bfa2cc14e
2018-11-08 12:34:36 -08:00
Jun Wu
4a49dc876f crecord: fix line number in hunk header
Summary:
`@@ -1,1 +-1,0 @@` is not a valid patch hunk header.
Change it to `@@ -1,1 +0,0 @@`.

This is a backport of my upstream patch https://phab.mercurial-scm.org/D3737

Reviewed By: singhsrb

Differential Revision: D12970975

fbshipit-source-id: eb769e93076efc932a8c4be7387ba21933aae331
2018-11-08 12:34:36 -08:00
Saurabh Singh
b481467dd4 doc: fix 'make doc'
Summary: This has been broken after D12832280.

Reviewed By: quark-zju

Differential Revision: D12981581

fbshipit-source-id: 3f411260e24e623abb6f18814fa5b0bf87b416cd
2018-11-08 10:46:24 -08:00
Kostia Balytskyi
d2c2c250e0 windows: use correct rename exception raising logic
Summary:
The way I wrote this, `if e.errno != errno.EEXIST or e.errno != errno.EACCES`
this condition never succeeds. So we're re-raising always, rather than falling
back to a `delete/rename` strategy.

Differential Revision: D12974035

fbshipit-source-id: 0b500d009315ba0094019a82576d5a3698b624b2
2018-11-08 06:00:16 -08:00