Summary:
Unfortunately required symbols are not exposed by lz4-sys. So we just declare
them ourselves.
Make sure it compresses better:
In [1]: c=open('/bin/bash').read();
In [2]: from mercurial.rust import lz4
In [3]: len(lz4.compress(c))
Out[3]: 762906
In [4]: len(lz4.compresshc(c))
Out[4]: 626970
While it's much slower for larger data (and compresshc is slower than pylz4):
Benchmarking (easy to compress data, 20MB)...
pylz4.compress: 10328.03 MB/s
rustlz4.compress_py: 9373.84 MB/s
pylz4.compressHC: 1666.80 MB/s
rustlz4.compresshc_py: 8298.57 MB/s
pylz4.decompress: 3953.03 MB/s
rustlz4.decompress_py: 3935.57 MB/s
Benchmarking (hard to compress data, 0.2MB)...
pylz4.compress: 4357.88 MB/s
rustlz4.compress_py: 4193.34 MB/s
pylz4.compressHC: 3740.40 MB/s
rustlz4.compresshc_py: 2730.71 MB/s
pylz4.decompress: 5600.94 MB/s
rustlz4.decompress_py: 5362.96 MB/s
Benchmarking (hard to compress data, 20MB)...
pylz4.compress: 5156.72 MB/s
rustlz4.compress_py: 5447.00 MB/s
pylz4.compressHC: 33.70 MB/s
rustlz4.compresshc_py: 22.25 MB/s
pylz4.decompress: 2375.42 MB/s
rustlz4.decompress_py: 5755.46 MB/s
Note python-lz4 was using an ancient version of lz4. So there could be differences.
Reviewed By: DurhamG
Differential Revision: D13528200
fbshipit-source-id: 6be1c1dd71f57d40dcffcc8d212d40a853583254
Summary:
Use the newly added zero-copy method to improve Rust lz4 performance. It's now
roughly as fast as python-lz4 when tested by stresstest-compress.py:
Benchmarking (easy to compress data)...
pylz4.compress: 10461.62 MB/s
rustlz4.compress_py: 9379.41 MB/s
pylz4.decompress: 3802.85 MB/s
rustlz4.decompress_py: 3975.61 MB/s
Benchmarking (hard to compress data)...
pylz4.compress: 5341.69 MB/s
rustlz4.compress_py: 5012.30 MB/s
pylz4.decompress: 6768.17 MB/s
rustlz4.decompress_py: 6651.08 MB/s
(Note: decompress can be visibly faster if we return `bytearray` instead of
`bytes`. However a lot of places expect `bytes`)
Previously, the result looks like:
Benchmarking (easy to compress data)...
pylz4.compress: 10810.05 MB/s
rustlz4.compress_py: 11175.36 MB/s
pylz4.decompress: 3868.92 MB/s
rustlz4.decompress_py: 634.56 MB/s
Benchmarking (hard to compress data)...
pylz4.compress: 4565.91 MB/s
rustlz4.compress_py: 622.94 MB/s
pylz4.decompress: 6887.76 MB/s
rustlz4.decompress_py: 2854.79 MB/s
Note this changes the return type from `bytes` to `bytearray` for the
`compress` function. `decompress` still returns `bytes`, which is important for
compatibility. Note that zero-copy `bytes` can not be implemented `compress` -
the size of `PyBytes` is unknown and cannot be pre-allocated.
Reviewed By: DurhamG
Differential Revision: D13516211
fbshipit-source-id: b21f852c390722c086aa2f37a758bf3f58af31b4
Summary:
This is intended to replace the python-lz4 library so we have a unified code
path.
However, added benchmark indicates the Rust version is significantly slower
than python-lz4:
Benchmarking (easy to compress data)...
pylz4.compress: 10964.14 MB/s
rustlz4.compress_py: 12126.00 MB/s
pylz4.decompress: 3908.29 MB/s
rustlz4.decompress_py: 798.68 MB/s
Benchmarking (hard to compress data)...
pylz4.compress: 5615.86 MB/s
rustlz4.compress_py: 740.32 MB/s
pylz4.decompress: 6145.68 MB/s
rustlz4.decompress_py: 2423.99 MB/s
The only case where the Rust version is fine is when the returned data is
small. That suggests rust-cpython was likely doing some memcpy unnecessarily.
Reviewed By: DurhamG
Differential Revision: D13516207
fbshipit-source-id: 72150b15c38bc8d8c7e7717a56a41f48d114db19
Summary:
When pulling an infinitepush bundle from the server to a client, it
uses remotefilelog's excludepattern functionality to force the server to serve
filelogs to the client. Unfortunately, it didn't specify what type of pattern,
and therefore it was treated as a regular expression. This meant that any path
that happened to be a regular expression would generally not match itself,
resulting in data not being sent to the client.
The fix is to just prefix these with 'path:' since we know they are all exact
paths.
Reviewed By: phillco, quark-zju
Differential Revision: D13521814
fbshipit-source-id: 8afc35b37d5858913b80ed0babfa0b5e401f0ab4
Summary:
If a filename is a valid regex and it doesn't match itself, that file
data will not be delivered to the client when doing a infinitepush pull that
requires a rebundle. This adds a test. The next diff fixes it.
Reviewed By: phillco
Differential Revision: D13522579
fbshipit-source-id: 6f8c5df20c31b834a33dbb9b73dc9031e26e969b
Summary:
Sandcastle jobs are using mercurial in conjunction with Phabricator CATs as a part of Phabricato security effort.
Here is an example of a mercurial command that failed because of this:
https://our.intern.facebook.com/intern/sandcastle/job/1498468091/
This adds logic to support parsing CATs from .arcrc for mercurial commands
Reviewed By: quark-zju
Differential Revision: D13468442
fbshipit-source-id: 033806d0e0779f9e7ade054d21e4cdbbdef08ed0
Summary:
Tests don't close the child process stdout. On newer versions of Python, this
can lead to ResourceWarnings when the test runner thread terminates.
Reviewed By: HarveyHunt
Differential Revision: D13517425
fbshipit-source-id: 6cedf4f39efe1299c41dbde784daf8c159309640
Summary:
Some of the escape sequences in run-tests.py are invalid. These cause
DeprecationWarnings on newer versions of Python.
In both cases, there are `\` characters that need to be escaped as `\\`.
Reviewed By: HarveyHunt
Differential Revision: D13517137
fbshipit-source-id: a899c3c28d55210f5972a515474a2fa69d051671
Summary:
The duringundologlock config and hook were used to test the undolog using
timing. We've replaced that, so remove the hook.
Reviewed By: quark-zju
Differential Revision: D13504644
fbshipit-source-id: a6b5fb308bc8938eec72788d93c9be6c237b72d7
Summary:
The undo tests use timing to detect when the lock is being taken. This is
flaky. Instead add extra logging to detect when the lock is taken.
Reviewed By: quark-zju
Differential Revision: D13504643
fbshipit-source-id: 07b80e416047d11b4ba3e1631c2385e5f12fa36f
Summary:
In a number of places, the remotefilelog code needs the information for
a single node but instead fetches the entire ancestor set. Now that we have
getnodeinfo, let's switch to using that. This was a primary hot spot in hg push
of large stacks of commits.
Reviewed By: quark-zju
Differential Revision: D13460746
fbshipit-source-id: 3aa288c70c87dcb32c0404311f27bbc87ddc5267
Summary:
Be compatible with `hgext.` or `hgext/` prefix. But print a warning saying it's
deprecated.
Reviewed By: DurhamG
Differential Revision: D13490362
fbshipit-source-id: ef13bd57a74be810df409af18a6259bc7b2b6dad
Summary:
This fixes LFS compatibility with packlocaldata.
Also drop the config as the default version is 1 now.
Reviewed By: DurhamG
Differential Revision: D13469486
fbshipit-source-id: 1dc4a1051667419d7aab97bf95f93cacd166468a
Summary:
It's meaningless to test the fetch cost in .t tests. So let's just set it to 0.
This also makes the output stable. So `run-tests.py -i` can be used directly.
Reviewed By: DurhamG
Differential Revision: D13469488
fbshipit-source-id: a2cf900531f3ff0f58957bd79d7283928b5da700
Summary: Previously, the test wasn't using a remotefilelog repo.
Reviewed By: DurhamG
Differential Revision: D13469487
fbshipit-source-id: 3c85633b8bfc209fa0d0317ca05247e14faf689d
Summary:
The test failed because "hgext" is no longer a namespace package. The check
becomes unnecessary since we almost always use the in-repo modules.
Reviewed By: DurhamG
Differential Revision: D13453160
fbshipit-source-id: 0c5a72b3abeb0a12426d8ed15c045a4ce478a4c6
Summary:
Add a new `--print` option to drawdag which prints the commit hashes and
descriptions of all the nodes that were generated by drawdag.
Reviewed By: quark-zju
Differential Revision: D10149266
fbshipit-source-id: 55b4c133b3c98c0258419811d7c00a3ec73a02cc
Summary:
Update drawdag to generate mutation metadata in commits when mutation recording
is enabled.
In order for this to work, we need to add the mutation relationships as edges
between the commits, so that when walking the DAG the predecessors are already
committed and we can find their hashes.
Reviewed By: ikostia
Differential Revision: D9989024
fbshipit-source-id: 671c1eb6c4ae6e87760efb4d3aa47e5e0585c94d
Summary:
Add support for mutation information being added when commits are histedited.
For the most part, the histedit predecessor and successor have a 1:1
relationship. The special case is for when commits are folded or rolled up.
In this case we act in the same way as a fold.
Reviewed By: quark-zju
Differential Revision: D9975466
fbshipit-source-id: 748040232f49aa87af4e25a97d948995d956f04a
Summary: Add support for mutation information being added when a commit is pushrebased.
Reviewed By: DurhamG, quark-zju, ikostia
Differential Revision: D9975558
fbshipit-source-id: 55444c05df32e5be159fbc19610f7a13ab103109
Summary:
Add support for mutation information being added when a commit is split.
The top of the final stack is the commit that stores the mutation metadata.
The other commits in the stack are just normal new commits, although the final
commit does record their hashes in its metadata.
Determining when the last commit in the stack happens before it is committed is
a little hard, as the code previously relied on the dirstate ending up clean
to detect a finished split. Instead we look at the patches that come out of
the filter function and see if they match the original patches that were sent
in.
Reviewed By: DurhamG, ikostia
Differential Revision: D9975469
fbshipit-source-id: acec485f9b561952f4ccdbaaf9491c9d48a70f58
Summary:
Add support for mutation information being added when a commit is metaedited or
folded.
The folded commit uses the top of the original stack as its predecessor. The
other commits that are folded are also stored in the metadata.
Reviewed By: DurhamG, quark-zju, ikostia
Differential Revision: D9975471
fbshipit-source-id: 92089d14cd5d65df283bbdc80a8b352b5f3982e9
Summary: Add support for mutation information being added when a commit is rebased.
Reviewed By: DurhamG, quark-zju, ikostia
Differential Revision: D9975481
fbshipit-source-id: 02a1ace0d6c80d73b1ab76e4d1b23a8d46ab049b
Summary: Add support for mutation information being added when a commit is amended.
Reviewed By: DurhamG, ikostia
Differential Revision: D9975472
fbshipit-source-id: 57d51823e709f71a88ad0fb86992a0a38541527c
Summary: Add a new command to debug-dump mutation information in commits
Reviewed By: DurhamG, quark-zju
Differential Revision: D9975474
fbshipit-source-id: a9ed1578dffc80da3e375837cede8d8685ee0427
Summary:
This is subtle.
`.hgignore`'s current behavior: if `^foo$` rule exists, then directory `foo/`
is ignored (ex. 'foo/bar' should be ignored).
However, that imposes problems for the sparse ignore matcher, which is the
"negate" of the "include" matcher. A user can write `[include]` glob patterns
like:
a*{b*,c*/d*}/e*
The ignore matcher will be the negate of the above patterns. Then because
`a1b2` does not match `a*{b*,c*/d*}/e`, the negate matcher returns "True",
and the ignore matcher will ignore the directory. So even if file `a1b2/e3`
should be selected, the parent directory being ignored cause the file to
be ignored.
That is clearly incorrect for sparse's usecase.
I think the issue is fundementally a layer violation - it's the *matcher*'s
responsibility to check whether one of the parent directory is matched (or
ignored), not the directory walker's responsibility.
This diff fixes the walker so it uses the visitdir interface, and moves back
the directory check to hgignore matcher to maintain compatibility. For three
matchers involved in ignore handling:
- hgignore matcher: updated to do the recursive directory handling on its own
- gitignore matcher: work out of box. already consider parent directories!
- sparse matcher: want the new behavior
`test-sparse-issues.t` is now green.
With this change, the `forceincludematcher` subdir hack used in sparse is no
longer necessary. Therefore removed.
Besides, all ignore matchers can handle "visitdir" correctly. That is, if
`visitdir('x')` returns `'all'`, then `visitdir('x/y')` will also return `'all'`.
Therefore the parent directory logic in `dirstate.dirignore` becomes
unnecessary and dropped.
Reviewed By: DurhamG
Differential Revision: D10861612
fbshipit-source-id: aa0c181ae64b361b85f08b8fecfdfe6331e9a4c2
Summary:
The script was used to check dependency on extenral extension. It's no longer
necessary since those "external" extensions are in repo now.
Reviewed By: DurhamG
Differential Revision: D13440398
fbshipit-source-id: 2f228ddee621b84a23f44748d843f29e1f05530e
Summary:
The extension was mainly for users `pip install <3rd-party-hg-ext>`, which
usually installs them into `hgext3rd`. Since we no longer search into
`hgext3rd`, this extension can be removed.
Reviewed By: DurhamG
Differential Revision: D13440401
fbshipit-source-id: d67a655ece9a1f6d5feb18d8aeac84852edfdcb9
Summary:
Our codebase is already unfriendly to 3rd-party extensions and we will likely
break the compatibility as part of revisising / replacing components in Rust
anyway. So just stop supporting `--config extensions.3rdpartyfoo=`.
Note `--config extensions.3rdpartyfoo=foo.py` still works.
This also makes `ImportError` more meaningful. Previously, when `hgext.foo`
imports `bar` which does not exist, the error is "cannot import foo", while
it should really be "cannot import bar". That is because extensions.py falls
back to import `hgext3rd.foo`, and `foo`. Now extensions.py only tries
`hgext.foo` and will suface the right error.
Reviewed By: DurhamG
Differential Revision: D13440400
fbshipit-source-id: c659f6d05abbc0d09f2811f36c0dc31e49491f19
Summary:
In an upcoming diff we'll be making hgsubversion pull use a single
transaction. We need the lastpulled file to be part of that transaction. The
easiest way to do that is to store it in the database.
Reviewed By: quark-zju
Differential Revision: D13433558
fbshipit-source-id: 013ce09aa7a7382008979a7f424860fc98dc8b5d
Summary:
As the title. This exposes the next issue, which appears to be caused by
`forceincludematcher` not working with complex patterns.
Reviewed By: DurhamG
Differential Revision: D10861613
fbshipit-source-id: d58c74fdf5da2b0399fe69ca499169cd5887645f
Summary:
Add a test about NEED_CHECK cleanup for files outside sparse. It does not
demostrate anything wrong, unfortunately.
Reviewed By: DurhamG
Differential Revision: D12906651
fbshipit-source-id: 6e9e690ba431e141666ffdb431f62280e86bbf0b
Summary:
I faced an issue where I had the following DAG:
```
A B
@ o
| /
| /
|/
o C
```
I did the following operations at that point:
```
hg hide -r A
hg up top
```
but the update took me back to A instead of taking me to B as I expected.
quark-zju debugged the issue and we found out that this was due to D6899232.
This commit fixes the issue by ensuring that we do not make a filtered repo
unfiltered before the update operation.
Reviewed By: DurhamG
Differential Revision: D13388494
fbshipit-source-id: 1678f97a4b3179720c30cf1366e9301a2c27a5ba
Summary:
I faced an issue where I had the following DAG:
```
A B
@ o
| /
| /
|/
o C
```
I did the following operations at that point:
```
hg hide -r A
hg up top
```
but the update took me back to A instead of taking me to B as I expected.
This commit just adds a test case that captures this issue.
Reviewed By: quark-zju
Differential Revision: D13389113
fbshipit-source-id: 6febb5459db3fbb69bbb7d23b4f0c6e1a12d4130
Summary:
Now that git hashes are stored in the commit extras, let's add a
command that reads the latest commits in the repo and adds their git/hg mapping
to the map.
Reviewed By: quark-zju
Differential Revision: D13369867
fbshipit-source-id: 250a62979472c2ab68b2ca6f6c6463daca4bdb45
Summary:
The name "meta" used in remotefilelog.py is confusing. It can sometimes be
"filelog meta", sometimes "datapack meta". The old code passes the wrong
meta and caused issues.
Differential Revision: D13380634
fbshipit-source-id: a1a6ada2d880b6c28fe22c486c45a6203e4cf5db
Summary:
This requirement was discussed in both this thread:
https://fb.facebook.com/groups/247583349136387/permalink/359377874623600/ and
D13277462. Basically, we want
```
hg log -r <hash> -T "globalrev()"
```
to return the svn revision in case there is no global revision assigned to the
commit which is possible for older commits.
Reviewed By: phillco
Differential Revision: D13370832
fbshipit-source-id: 5f8ba4c1781f83204de775127554a4944f00bb1d
Summary:
Add a config option to use indexedlog as the node map storage instead
of a flat text file.
Reviewed By: quark-zju
Differential Revision: D13062573
fbshipit-source-id: ae14df24a4e36c59fbd9ec82d785aac52a2f8b5f
Summary:
Backports my upstream patch (https://phab.mercurial-scm.org/D2289)
Discovered by nemo on #mercurial freenode IRC, some months did not parse.
Reviewed By: DurhamG
Differential Revision: D12983571
fbshipit-source-id: b33227b0c986180966579afbb2b7c11fd48d275b
Summary:
In order to move our hg-git mirroring off of the main hg servers, we
need to make it possible for the hg servers to compute the hg-git mapping
without having the entire git repository available. To do so, let's store the
git hash as an extra in the hg commit.
This breaks bidirectionality, but we've long since not needed that.
Reviewed By: phillco
Differential Revision: D13362980
fbshipit-source-id: 51df709bc5e77d78bb963abf90d0c35bb743d966
Summary:
This bug has been here for 2+ years. Basically, when gathering the
ancestors for a given file node, if it traversed a rename it would lose track of
the new name and instead look up the new hash by the old name, which would fail.
We didn't hit this often, because it only causes a problem when you have partial
history and have to go fetch more history. In most remotefilelog repos we
download all of history, so you always have everything and therefore never hit
this.
Reviewed By: kulshrax, singhsrb
Differential Revision: D13332459
fbshipit-source-id: 120bfe9ac618a4979e1685f24dc6462fc7415b1b
Summary:
Adds a test that demonstrates a bug in our ancestor traversal logic.
The test requires very particular circumstances:
Assuming a file x was modified to x' then renamed to z:
1. The rename is required because the bug is in keeping track of the copy source name
2. Version x' must be in the same pack as version z, because the bug is in the
logic that traverses a single packs history.
3. Version x must be in a different pack than x' and z, because the bug is about
trying to look up version x after having lost track of the correct name when
going from z to z'.
4. We must be using rust historypacks, because that implementation attempts to
traverse the entire history within the pack, while the python implementation
stops at renames.
Reviewed By: kulshrax
Differential Revision: D13332458
fbshipit-source-id: b01d09fb8ebd27414e4f6dba06a6d0b1f26ac13c
Summary:
Oculus wants to preserve the "do not show binary diff" config while running
`hg export` in automation, since printing the binary diff might just OOM the
container. Let's add a plain exception for it.
Reviewed By: zhh95
Differential Revision: D13111212
fbshipit-source-id: 34af58ac0917de3b3231e637774896d882585e26
Summary: As the title. To help decide the right way for D13332653.
Reviewed By: DurhamG
Differential Revision: D13333157
fbshipit-source-id: 4fe44fffa48409e790efc1ba7b0d681350512eac
Summary:
We've seen corrupted repository in the wild on which we were unable to garbage
collect. Let's add a test to verify that the other repositories will be garbage
collected.
Reviewed By: DurhamG
Differential Revision: D13328487
fbshipit-source-id: 6f6a69d14455d00e468c5b9186d65cca8fd5c1c3
Summary:
In a future diff we want pushrebase to be able to rewrite commits
pulled from svn. In that situation, we need to be able to rewrite the svn
metadata so that the post-pushrebase commits are mapped to the svn rev numbers.
Let's add a truncatemeta command that will walk the latest commits in the
changelog and redo any that aren't mapped correctly.
Reviewed By: markbt
Differential Revision: D13074960
fbshipit-source-id: fe98879dd16cc0806d20ef6eab5f9d77f0e0c877
Summary:
D13267528 malformed the bookmark store files for testing by deleting a
random file. This makes the test flaky since sometimes a non critical file is
deleted and the exception is not raised. Therefore, lets malform the store more
reliably by truncating all the files in the store.
Reviewed By: ikostia
Differential Revision: D13271362
fbshipit-source-id: 684ffbcdc59e943476a6acc2c4152bf65cc12e1f
Summary:
Importing anything from python files with explicit breaks embedded
aproach we use on Windows.
For one of the tests, I did add `hg debugshell -c "from hgext import sparse;
print sparse.__file__` just to make sure the same file is used in practice.
Reviewed By: singhsrb
Differential Revision: D13258984
fbshipit-source-id: 4c86e1dccfd4f525a14019dc4b9586d798f2bffe
Summary:
D13133605 changed the python bookmark store API and broke the tests.
To fix, I have migrated the tests to use the new API. Also, since most of the
tests was just python code, I have moved the test from being a `.t` test to a
`.py` test.
Reviewed By: quark-zju
Differential Revision: D13267528
fbshipit-source-id: 69b5a7110f63ea60e1bac402064edbffbf64420f
Summary:
If full is specified, or the maxage config option changes from one sync to
another, we need to actually pull in the new commits. Since the version number
won't necessarily have changed, we need to request a full copy of the cloud
workspace to work from.
Reviewed By: liubov-dmitrieva
Differential Revision: D13214274
fbshipit-source-id: b21594c04c05f065caf9f9dc494e6274debbde5c
Summary:
Stop comparing the local heads and bookmarks with the full cloud heads and
bookmarks when some heads or bookmarks have been omitted. Since they're
omitted, they won't be there.
Reviewed By: liubov-dmitrieva
Differential Revision: D13214275
fbshipit-source-id: 35a897f053f58d0793d384ff60b8202e80aec0c7
Summary: Since we are using `hg.rust` with everything, we should also test it.
Differential Revision: D13121588
fbshipit-source-id: 3a4e07c1d70c279f008812d026bf9a7c74aa3b2e
Summary:
This had a nasty bug where it manually computed the cached base text
using the full manifest text. When operating in a treeonly repository, this
resulted in corrupt deltas. The only reason we didn't hit it earlier is because
the secondary stackpush path was added and doesn't have this issue. But it
couldn't handle merge commits, which is why we hit this when it came time to add
a merge commit.
Now that the main repo that needed this optimization is treeonly, we don't need
this precaching anymore.
Reviewed By: mitrandir77
Differential Revision: D13154342
fbshipit-source-id: 5cb74167665a60a36e4a6d926f3f9f1c5c7bbef1
Summary:
There's a corruption that occurs because pushrebase caches a fulltext
instead of a tree. Let's write a test to expose it. The next diff will fix it.
Reviewed By: mitrandir77
Differential Revision: D13154341
fbshipit-source-id: e2ac040c59c677b414b6eddd39d7dc2ebb25f082
Summary:
Fix tests.
This makes sure that the grep process itself is not matched.
Reviewed By: farnz
Differential Revision: D13138516
fbshipit-source-id: d29829584c6fe34dd60345b2812682d4cb9d51e4
Summary:
When we have omitted syncing commits from the cloud workspace, give the user
advice on what to do to fetch them if they run `hg sl --all`.
Reviewed By: markbt
Differential Revision: D13085505
fbshipit-source-id: 624ce6cbf4cb2194ab5ffbc09c1ac3e073932249
Summary: Now that `chg` is embedded into `hg.rust`, we need to explicitly disable it while tests run.
Reviewed By: quark-zju
Differential Revision: D13021560
fbshipit-source-id: 843045f3bc9d9866e421f0bf0aa132849e5b338f
Summary:
Add "--dry-run" for fix-code.py and use it in test-check.
This avoids license header and version = "*" issues.
Reviewed By: ikostia
Differential Revision: D10213070
fbshipit-source-id: 9fdd49ead3dfcecf292d5f42c028f20e5dde65d3
Summary:
Add a new config option: `commitcloud.max_sync_age`. When set, commit cloud
will not pull in any commits that are older than this when it is joining or
syncing. The commits are still nominally in the cloud workspace, we just
save join or sync time by not including the commits.
Reviewed By: liubov-dmitrieva
Differential Revision: D13062470
fbshipit-source-id: 17a4bdb4095766a83a4bf6d4151ae86b39edf59c
Summary:
tweakdefaults used to do this if configured.
After dicussion with the team, we decided that returning 1 was not a helpful design decision, so we're going to fold this in here and everywhere.
Reviewed By: singhsrb
Differential Revision: D13050548
fbshipit-source-id: 66e834ea503e4b1339e369495a9729b951024a6d
Summary:
This allows us to expose `arc pull`'s functionality directly from hg.
There are three parts:
- Create a revset that runs a program, reads the stdout and looks up that commit if it exists
- Auto-pulling if the commit doesn't exist
- Supporting an optional argument (target) that's passed to the script
Reviewed By: DurhamG
Differential Revision: D10524541
fbshipit-source-id: 7493c5592e272f9e8a87f109cec1426d44935ecc
Summary:
We need to record updated onto rev instead of guessing it in pushrebase replayer.
I will not land this diff until mononoke db schema is update D12923144, D12997125
Reviewed By: quark-zju
Differential Revision: D12922833
fbshipit-source-id: 11c6411c392ca9092be53ffba8baa074faf3a996
Summary:
Previously, if we were writing local data to pack files we weren't able
to read that data until the pack file had been flushed. Let's add those mutable
packs to the union store so we can see there data.
This is important for unblocking use of hggit and hgsubversion since they may
import many dependent commits in a single transaction.
Reviewed By: quark-zju
Differential Revision: D12959497
fbshipit-source-id: 405c0c5c1e8fc84bc8ffef827a84e91d57eb95d8
Summary:
The config to write local packs has an issue with reading data from
currently pending mutable packs. Let's add a test demonstrating this, and in the
next diff we'll fix it.
Reviewed By: quark-zju
Differential Revision: D12959501
fbshipit-source-id: 5bcb278ccb977cea83e8cd594a404816d4f8f1fa
Summary: Jellyfish uses `hg debugcrdump` to get commit data, including binary files. Now we want in jf to be able avoid dumping binary files, so I added `--nobinary` option to crdump
Reviewed By: quark-zju
Differential Revision: D12981968
fbshipit-source-id: 669ef4cf8f6225b911f5085c1a47fb7242f259bf
Summary:
`@@ -1,1 +-1,0 @@` is not a valid patch hunk header.
Change it to `@@ -1,1 +0,0 @@`.
This is a backport of my upstream patch https://phab.mercurial-scm.org/D3737
Reviewed By: singhsrb
Differential Revision: D12970975
fbshipit-source-id: eb769e93076efc932a8c4be7387ba21933aae331
Summary: Updated the global help summaries that are displayed for each command when you run 'hg help' and fixed corresponding tests
Reviewed By: markbt, kulshrax
Differential Revision: D12832280
fbshipit-source-id: 950dad1c805feab573d7d0182da523ae12299d3b
Summary:
Our `rename` is not atomic:
```
def rename(src, dst):
"""Rename file src to dst, replacing dst if it exists"""
try:
os.rename(src, dst)
except OSError as e:
if e.errno != errno.EEXIST:
raise
unlink(dst)
# What if the process is interrupted here?
os.rename(src, dst)
```
However, the `MoveFileEx` Windows API provides a way to to
replace the existing file, thus eliminating the need to do `unlink`.
Unfortunately, it only works for files, not for dirs, therefore
we're introducing a new file-specific rename function.
Differential Revision: D12940555
fbshipit-source-id: a6749a9b16a285788de0f5c06d51a15c919166ce
Summary:
If the commits being pushrebased don't actually need to be rebased, pushrebase
will accept them like a normal push. In this case we shouldn't prevent them
from be changed to public commits on the client.
Track this by detecting whether we receive a changegroup part from the server.
If we do, only the commits that are marked as replaced with obsmarkers (if
enabled) can be marked as public.
Reviewed By: quark-zju
Differential Revision: D12944851
fbshipit-source-id: 44f8fc17b36397d949cba5d3e787fad813bab4ea
Summary:
When pushrebase is enabled, pushing a commit that doesn't result in any
rebasing action acts like a normal push (the commit is not modifed and a new
hash is not returned), however the phase update doesn't apply, so the commit
remains draft.
Reviewed By: quark-zju
Differential Revision: D12944850
fbshipit-source-id: 2145d9b7eebc27bfdff34544e73802c13f30dbd2
Summary:
D12888964 added the capability for `svnrev` revsets to be
interoperable with the `globalrev` revsets. This commit adds the tests for it.
Reviewed By: quark-zju
Differential Revision: D12888960
fbshipit-source-id: 07b12632d534a329c32179e978b058b1e427e688
Summary:
The capturing UI is not required as we can use the much cleaner
`ui.pushbuffer`/`ui.popbuffer` for the intended purpose.
Reviewed By: quark-zju
Differential Revision: D12906912
fbshipit-source-id: ae93e6417135ca23b28a11af63745464344fa3cf
Summary:
This method always prints the graph for the commits in the repository.
This commit adds supports for specifying any revset as input for the log and
also, makes printing the graph optional.
Reviewed By: quark-zju
Differential Revision: D12888965
fbshipit-source-id: 794606fa17f9836fd73675d36d11220b11994f41
Summary: The `svnrev` is more useful than the `rev` in this context.
Reviewed By: quark-zju
Differential Revision: D12888963
fbshipit-source-id: a902c57ecc3bc3eca9da77ab6e14ef58512d5ad8
Summary:
This is required for testing `globalrevs` integration with
`hgsubversion`. In particular, we will test out whether we can resolve
`r<svnrev>/r<globalrev>` correctly later. Seems like this configuration should
be on during all the tests because it is the configuration we use in
production. Therefore, this commit in resolves revset string with prefix `r`
for all the `hgsubversion` tests.
Reviewed By: quark-zju
Differential Revision: D12888961
fbshipit-source-id: af9b2e1aea9771f42fe1fb2d3f9a759a926b1ce3
Summary:
We need a configuration option for specifying the starting revision
beyond which we will consider the global revisions associated with the commit
valid for the following reasons:
- It lets us have the flexibility to support `globalrevs` from a commit of our
choice.
- We need this configuration to decide when we should fallback to looking at
the `svnrev` instead of the `globalrev`.
Reviewed By: quark-zju
Differential Revision: D12888966
fbshipit-source-id: 46d6314886ed5074edb537ba340d5606ec354619
Summary:
It turns out data packs contain the exact mercurial file blob,
including the rename metadata header. This is different from how loose files work, which
contain the non-headered text and instead store the rename in the history at the
end of the file.
Ideally we'd change datapacks to not store the metadata version of the file, but
unfortunately this is how it has always been, so instead we must change the
commit-straight-to-pack code to write the headered blob.
Differential Revision: D12936247
fbshipit-source-id: 3f909d2964d7f7200ac0e31e47bd195b0c9b03e5
Summary: Not all of the repos are remotefilellog, so I moved some invocagions of the content store to shallowrepo and added requirements check where the transaction is processsed in localrepo
Reviewed By: quark-zju
Differential Revision: D12881916
fbshipit-source-id: 1119debd1e3d42cdf9513a6cae4bad4c000cc046
Summary:
Based on discussions about the best approach for solving the amend message
problem, back out the template previously added. We will use a different
approach.
Reviewed By: mitrandir77, liubov-dmitrieva
Differential Revision: D12921753
fbshipit-source-id: ca760ffe14bfe473b7526a1b84a8cfc6b0257bf2
Summary:
It turns out tracking ignored files does have an impact on status performance.
Filtering out the ignored directories is not that fast, and ignored files can
be scattered everywhere (like ".pyc", ".iml" files) that makes them harder to
be filtered out efficiently.
Add code paths to migrate between "ignored tracked" and "ignore untracked".
Store the metadata in treestate.
Reviewed By: phillco
Differential Revision: D12916021
fbshipit-source-id: e02d0c6f3b1a036f70703c11f35381c594e2f8e5
Summary: This is to replace `debugstate -v`. The latter does ont list untracked files.
Reviewed By: DurhamG
Differential Revision: D12906649
fbshipit-source-id: b84f41dfadff4932c0ddd48480a4aa03db88cab0
Summary: We'd like to identify which of the three underlying conditions cases this exception.
Reviewed By: quark-zju
Differential Revision: D12870572
fbshipit-source-id: 45724ad6bb3582c04a6ea3c0b7a748bf72219b93
Summary:
This test is broken after D12849852 because the grep outputs the files
in different order during multiple test runs. Let's fix it by sorting the
output from the command.
Reviewed By: phillco
Differential Revision: D12869810
fbshipit-source-id: 49073e83e926e94db5f0bd290d74845ebfde0bb6
Summary: This is nicer for the user and makes the test not be flakey.
Reviewed By: quark-zju
Differential Revision: D12860852
fbshipit-source-id: 6ccb0ca679dc25a80730e83a26a39dd3becb5c8f
Summary: This fixes the "hg prev" targetting the first commit in the repo. The problem was the rev `0` which casted to false in all the checks.
Reviewed By: phillco
Differential Revision: D12840237
fbshipit-source-id: 594a2725fa125ee713f347379c875ec54b80a53d
Summary:
This brings the config closer to what we use and uncovers some problems with
disallowemptyupdate itself.
Reviewed By: phillco
Differential Revision: D12840239
fbshipit-source-id: 77740b79114c7d901b0181e0e425aea4e9c7c570
Summary:
a couple of problems:
* The recent move from tweakdefaults also broke `bin` symbol, rendering
this feature completely broken for non svn backed repos
* The `bgr` tool knows about fewer corpuses than we do, so go directly
to the underlying C++ client binary
* Add configuration options for that binary
Reviewed By: phillco, farnz
Differential Revision: D12849852
fbshipit-source-id: 154d4822d097602505349d3f67b45f19c17a7bf8
Summary:
The recent changes to the progress engine to work around locking bugs removed
the `clear` step of `progress.suspend`. This leads to garbled output when
a progress bar is ongoing. Restore it by adding a new lock method to the
engine.
Also add a `--with-output` option to `hg debugprogress` to test this.
Reviewed By: mitrandir77
Differential Revision: D12838415
fbshipit-source-id: 83ed516b528d0b0bbe37945141d50b50da00ac8e
Summary:
We need a way for mergedrivers to be able to read ancestor data (to properly detect a backwards merge, such as during `hg backout`).
Passing it directly is tricky since passing a new kwarg to functions without `**args` raises a `TypeError`. We'd have to update the hook framework to catch this `TypeError` and re-run the hook without the kwarg, which would be hard to do cleanly.
Rather, we can just add this as a transient property on `mergestate`, which seems like an appropriate place for it. Any callers trying to read this when it hasn't been set will get an exception. But there aren't any other users of this field since it hasn't existed before.
Reviewed By: quark-zju
Differential Revision: D12827614
fbshipit-source-id: 48dc84f1e60232625c8f2a73e81b5bc223c22bb3
Summary:
The "fsmonitor returned" file list was made short by D9997658. Do the same
thing for notefiles.
Reviewed By: phillco
Differential Revision: D10865333
fbshipit-source-id: ff31cde04210363e6af04e5578543f1b16310b11
Summary:
If a "normal" file is clean, drop the "NEED_CHECK" bit.
Not very sure how NEED_CHECK can be set for the entire working copy while
`_poststatusfixup` does get executed. But it happened. Fix it by explicitly
removing NEED_CHECK from clean files.
Reviewed By: markbt
Differential Revision: D10865256
fbshipit-source-id: aafd3195a8acf39178bbc2d5f919345a7320f2b7
Summary:
If the NEED_CHECK bit was set accidentally (reason yet to know), there is no
way to recover automatically by running "hg status" without touching the files.
Add a test to show that.
Reviewed By: markbt
Differential Revision: D10865255
fbshipit-source-id: 60b6914a9cd778e60815f83ba98c29c9c5f88b34
Summary:
This test still fails on OSX because the messages with prefix
`remote:` are missing. It doesn't seem like we need those messages for the
actual testing. Therefore, adding the `-q` option to the affected commands to
remove the `remote:` messages for all the platforms.
Reviewed By: quark-zju
Differential Revision: D12829679
fbshipit-source-id: e53dca96b5511049ef240ed671705987cdd9eab1
Summary:
When syncing a commit cloud repo, if the user has put a bookmark on a public
commit that we don't have locally, also pull that public commit into the repo
so that we can put the bookmark on it.
Reviewed By: quark-zju
Differential Revision: D12815471
fbshipit-source-id: 080038e4e239170cad994ff4ab9326d787d1c0fa
Summary:
If you put a bookmarks on a public commit that one of your other synced
repos doesn't have, it will delete the bookmark when it syncs. This deletion
is then propagated to all your other synced repos. This appears to the user
like the bookmark just vanishing.
Reviewed By: quark-zju
Differential Revision: D12815470
fbshipit-source-id: 802f85433b9ad15db3fb9e82e1222224eff4ca88
Summary:
Introduce a new template config option `amend.messagetemplate`, which, when
set, is applied to the commit message when amend is run with the `-m` option.
The template is provided with the old message, as `oldmessage` as well as the
message provided on the amend command line, as `message`. This can be
configured so that the amend message only overrides part of the old message.
Reviewed By: quark-zju
Differential Revision: D10857454
fbshipit-source-id: 6eb6f33624fae32fc7a344769e39f5c7a78a3604
Summary:
The `truncate` tool isn't availble on OSX, so include our own in the test
suite.
Reviewed By: DurhamG
Differential Revision: D12815613
fbshipit-source-id: 510b2936f07c5193671baaeaec6620872c3ec982
Summary:
Add label annotations to the blame template indicating how old the line is
(bucketed into various ranges). Define colors for these labels so that recent
lines are white, and older lines fade through yellow to black.
Reviewed By: phillco
Differential Revision: D10869068
fbshipit-source-id: 2b99c84c115c3f9e408468f52b5754ff820fcec5
Summary:
Make the space for Phabricator diff numbers larger, and right-justify the
output. This makes it look better.
Reviewed By: phillco
Differential Revision: D10868646
fbshipit-source-id: 7cf50677560a15ef77cc3681063e902d8d9e7a18
Summary:
This makes tests closer to production setup and removes a bunch of "saved
backup bundle to ..." messages.
With D9236657, this should not hurt server-side performance.
Unfortunately a lot tests cannot be migrated easily, mostly because revision
numbers are used. They are left with a TODO.
Reviewed By: DurhamG
Differential Revision: D9237694
fbshipit-source-id: c993fce18f07aba09f6d70964e248af8d501575a
Summary:
The test infra does not handle "skip" state gracefully and treats skips as
"failure". That generated scary reports like hundreds of tests are failing.
Stop reporting skips to fix it.
An enivronment variable is added to get the old behavior.
Reviewed By: singhsrb
Differential Revision: D12808196
fbshipit-source-id: 2356a1d6fdfe1d2991d28368787c7d2b7b525b6b
Summary:
With the obsmarker copying feature, rebasing revs containing obsolete
relationship would result in the markers being copied correctly.
Reviewed By: phillco
Differential Revision: D7121486
fbshipit-source-id: a666ad65c2eccbfd26db813d4f792c66489ead8c
Summary:
If a corrupt pack file is encountered while marking the ledger for repack,
handle the failure.
Don't immediately delete the pack files. Keep them around as we may be able
to repack some of the non-corrupt contents (particularly when the file has
been truncated).
Once repack has completed, revisit the corrupt packs that we found. For shared
packs, delete them. For local packs, rename them. They are the only source of
the data, and we may be able to recover more data manually.
Reviewed By: DurhamG
Differential Revision: D10527544
fbshipit-source-id: f228fb4de3ff016a1c9035823bfa1d6b7767cdcb
Summary:
The incremental repack implementation omits repacking local data - it was
recently added for full repack, but not incremental repack. Add it to
incremental repack.
We also don't run `_deletebigpacks` for non-shared data, as that data is
only stored locally.
Reviewed By: DurhamG
Differential Revision: D10525957
fbshipit-source-id: 012d7e8d455ac607773fc7013dcc4df0881665e8
Summary:
Loose files don't contain the original filename inside them, but rather the
directory name is based on the sha1 hash of the filename. This means that
given a remotefilelog data file, we can't find out the name of the file that
it contains.
Repack needs to know this (plus it's also useful for debug purposes), so add
an additional file in the directory, `filename`, which contains the original
filename of all the data files in the directory.
It follows that `sha1(read(filename)) == basename(dirname(filename))`
Reviewed By: DurhamG
Differential Revision: D10516851
fbshipit-source-id: aeeb1a116e0d50a134c274c3bb31c7c26aa8015c
Summary: D10435279 somehow landed with a green test run, but I suspect it broke this.
Reviewed By: DurhamG, quark-zju
Differential Revision: D10863085
fbshipit-source-id: 3f4e95c7838c39f5801430356b72901d70086173
Summary:
tweakdefaults renamed core hg's `grep` to `histgrep`, and added a `hg grep` that behaves more like `git grep`, searching the working copy. This diff folds those changes into core.
The config changes from
```
[tweakdefaults]
allowfullrepohistgrep = False
```
to
```
[histgrep]
allowfullrepogrep = False
```
Reviewed By: quark-zju
Differential Revision: D10435279
fbshipit-source-id: ad3d1da5824511a612111715e46119d70066050f
Summary:
One thing that makes dealing with merge conflicts difficult at Facebook is that it can be tough to know *why* you have merge conflicts.
Because rebase processes one commit at a time, it's clear which of *your* commits is the problem, but finding the "other" commit is surprisingly hard. Often we rebase onto `master`, and in a monorepo, `master` is often a random commit that has no relationship to your code at all.
This diff attempts to improve the situation a bit by listing all of the commits between the common ancestor and the destination commit that modified the file. The user can then identify which are likely culprits:
```
warning: 2 conflicts while merging b! (edit, then use 'hg resolve --mark')
3 commits might've introduced this conflict:
- [0942ca9aff3d] random commit 1
- [3ebd0a462491] foo bar biz
- [3e5843b4b236] blah blah blah
```
When the destination is public, this approach requires one loose file download (~0.5s?) per conflicted file (this pulls in the full filectx history DAG, so further downloads aren't needed), which is likely acceptable as it only runs on conflicts, and provides value to the user. But it's also configurable for users performing massive codemods.
I'll add metrics to gather the average number if matched commits and the time taken to generate.
This doesn't put this information into the conflict markers, so a lot of people (Nuclide users or the default `editmerge` users) won't see it. We need to think creatively about how to expose it to them.
The ideal solution would tell you which commits edited the actual conflicted section(s), not just those that touched the file.
This requires fast remotefilelog downloads (cc @[100000771202578:Arun] :)). Once we have those, we can construct a local `linelog` of the commits involved, and used this to identify precisely which commits edited the conflicted sections of the file.
Reviewed By: quark-zju
Differential Revision: D9816270
fbshipit-source-id: 04e08dea7a9429eaeab0d40310cd34355104bb74
Summary:
Normally pushbackup should not take the repo lock. To debug when this does
occur in the background, add a debugstacktrace for lock aquisition when
infinitepushbackup.bgdebuglocks is set.
Reviewed By: quark-zju
Differential Revision: D10508808
fbshipit-source-id: 3cb2c155bac88751122111d91bdc1f51b381f7ff
Summary:
The [bug 29988](https://bugs.python.org/issue29988) is, when Ctrl+C (or other
signal) interrupts a program between certain Python bytecode instructions,
`__exit__` might be skipped incorrectly.
A single signal can only skip a single `__exit__`. Therefore it can be
worked around by using a list of `with`s, like:
with c, c, c, c, c:
....
Make `c.__enter__` and `__exit__` do the real `__enter__` and `__exit__` at
most once. To use the bug to skip all 5 `__exit__`s, 5 signals need to be
delivered at 5 tiny windows, which is unlikely to happen practically. Besides,
we can increase the count of `c`s to become more confident.
Note this has to be implemented natively since pure Python code cannot
have a reliable way to record whether `__exit__` was called or not:
self._obj.__exit__(...)
# Insert SIGINT here
self._exit_called = True
The inner object has to be native too to make `__exit__` atomic. Currently,
it's restricted to only the `Condition` object.
Reviewed By: markbt
Differential Revision: D10850510
fbshipit-source-id: 5c523a7bce568509641f8870d7ea381c0a99975c
Summary:
The Rust implemenation has a smaller chance to have deadlock issues when
interruptted.
Reviewed By: markbt
Differential Revision: D10843400
fbshipit-source-id: af5614b9992577bde74cf54c5760d15d050aafbf
Summary:
Python 2's `threading.Condition` and `threading.RLock` implementation are in
pure Python. Part of `RLock.acquire` looks like (simplified):
def acquire(self, blocking=1):
me = _get_ident()
if self.__owner == me:
self.__count = self.__count + 1
return 1
rc = self.__block.acquire(blocking)
########## Here #########
if rc:
self.__owner = me
self.__count = 1
return rc
If an interruption (ex. SIGTERM) happens at "HERE". The lock would be in an
inconsistent state. And if some `finally` block, or `__exit__` in a context
manager tries to release the lock, it could deadlock.
Similar problems also apply to `release`, `_acquire_restore`, and
`_release_save`. Basically, `self.__owner`, `self.__count` and `self.__block`
(i.e. the real lock) cannot be guaranteed in a consistent state in pure Python
world, because interruption can happen before any Python bytecode instruction
(but not inside a single Python bytecode instruction).
Therefore the interruption-safe implementation cannot be done in pure Python.
Use Rust to rescue.
The added test `streetest-condint.py` has a high chance to reproduce the
deadlock issue with Python 2.
Python 3 has a native RLock implementation, which makes things better. The
"Condition" implementation is not native and I haven't checked whether it
is sound or not.
Unfortunately, as part of testing, I hit https://bugs.python.org/issue29988 and
confirmed from the Rust world. That is, `__exit__` is not guarnateed called (!!)
That means native implementations still have a chance to be wrong, and there is
no easy way to fix it. `streetest-condint.py` was then updated to expose the
issue more easily.
The implementation is better than Python 2 stdlib, though.
Reviewed By: markbt
Differential Revision: D10517920
fbshipit-source-id: 394c9050c512ce2a0f9743c28ccfafe0f560141a
Summary:
Adds the `grep.biggrepcorpus` option that links the repo to the appropriate
biggrep corpus to each of the repo specific configs that we maintain.
This allows us to know when a repo can be used with biggrep.
Use this option to automatically enable the use of biggrep when in an eden repo.
Reviewed By: quark-zju
Differential Revision: D10434568
fbshipit-source-id: a2c77d8c58b611f818d08725a6298dec1263bb8a
Summary: I don't see the point of this message, since resetting an unmarked head is perfectly fine. Nuke.
Reviewed By: markbt
Differential Revision: D10510231
fbshipit-source-id: ad0c389ddd5bc40f9d6648c7c52968c4abd91d38
Summary:
This changes the git-mapfile to be append only. It then makes gitlookup
find hashes by scanning from the end of the mapfile backwards. Since most
lookups are for recent commits, this should greatly speed up hg->git hash
lookups and remove the need to sort the mapfile before writing.
Reviewed By: singhsrb
Differential Revision: D10521208
fbshipit-source-id: 7073d3f4796674416ae05e0ac9f36035ca0835db
Summary: I think D10446929 might've broken this by changing the pull string such that there was no German translation for it.
Reviewed By: quark-zju
Differential Revision: D10520682
fbshipit-source-id: a39b5911295ee0d1f5eee3b307930cffa87c3562
Summary: Updated help text for hg pull and hg rebase
Reviewed By: markbt
Differential Revision: D10446929
fbshipit-source-id: a81e186398cda02f58df95e6f2613b3f7457d9ed
Summary: Its functionality was folded into core a while ago, so we can just call `log -G`.
Reviewed By: quark-zju, singhsrb
Differential Revision: D10509538
fbshipit-source-id: 5c16ff708072e9bebabaf3b8e1474091dfb35216
Summary:
When you get an error, let's print the number of conflicts in each file. This will give the user some sense of how much work they have to do.
The code change is entirely in `filemerge.py`, and `tests/test-merge-conflict-count.t` adds a new test.
Reviewed By: quark-zju
Differential Revision: D9815243
fbshipit-source-id: 1b73a1db293902ac7242997a7d6ae09478344068
Summary:
Remove extensions that are not used in prodution. These extensions are also
unlikely to be used in the future.
Reviewed By: ikostia
Differential Revision: D10473370
fbshipit-source-id: a936e30acd3ec4370434c583447942c6ee8d9b13
Summary:
Since we can now skip a large ignored directory efficiently, just track the
ignored files in treestate. This fixes the "unignore" correctness issue that
has been there for years.
It also makes it easier to implement a faster version of "status -i" or "purge
--all". But that needs some extra changes.
It looks fine for fbsource sparse profiles. But we don't control all the
possible sparse configs out there. It's possible for certain users to have
sparse configs that cannot be optimized efficiently and their hg performance
will degrade. To reduce risk, a config option is added so we can slowroll the
feature and collect feedback.
Reviewed By: DurhamG
Differential Revision: D10352851
fbshipit-source-id: fbcb8445d87a50826852b832bbb0559ab199192f
Summary:
Previously, git-getmeta would have to scan through the entire git map
file, which was extremely expensive. Let's shift this cost to write time by
sorting the map file before writing, then using bisect to lookup nodes.
A while ago we switch hggit to only serialize the mapfile every 100 commits or
so, so the additional time here shouldn't be excessive.
Reviewed By: quark-zju
Differential Revision: D10470293
fbshipit-source-id: 47e23d251adde5456e7b593bfcc5976259a497f2
Summary:
Replace `sparse delete` (which deletes an include or exclude rule) with commands
that better describe what they do.
Also update `include` and `exclude` to additionally remove the rule from the
opposite set. It makes no sense for a file to be both included and excluded.
Reviewed By: phillco
Differential Revision: D10446704
fbshipit-source-id: 8966f7d0e54b1aae737fa0bae9cf35ed7ce88696
Summary:
Add a new sparse show command, which shows the current sparse configuration.
Use common templating, including colors, between sparse show and sparse list.
Reviewed By: phillco
Differential Revision: D10446705
fbshipit-source-id: acb539f80f625945716758c785703b46a81aa6f1
Summary:
Allow commands that have subcommands to categorize the subcommands. These
categories appear in the help for the top-level command.
Reviewed By: phillco
Differential Revision: D10446708
fbshipit-source-id: a7f2a9bc79c2a2c4df90517a5e6c9a4b78b31a72
Summary:
All ssh stderr are prefixed by "remote:", which could be confusing because the
message can also be some errors from the local ssh program. Treat "ssh:" as a
special case and do not prefix it with "remote:".
Reviewed By: markbt
Differential Revision: D10437082
fbshipit-source-id: 37935bd7276969ef17fa5028b02d01e3c9d0bf30
Summary:
On OSX the path to the hgrc can be so long it word wraps. Let's update
the test to accomodate the extra line.
Reviewed By: quark-zju
Differential Revision: D10453508
fbshipit-source-id: 5ba9db86f8475ab31ddb4b0e32da4feeeb8b5ce8
Summary: Signifying the change of perftweaks from an extension to a config grouping.
Reviewed By: quark-zju
Differential Revision: D10416857
fbshipit-source-id: b233a353c7da1d211ad0a7e53b942244863a83b3
Summary: Config-gate it so we don't have it popping up in tests everywhere.
Reviewed By: quark-zju
Differential Revision: D10432784
fbshipit-source-id: 531007432d67fef6ff504e319e43fb596d6a0d27
Summary:
Per D8985248, this code isn't necessary anymore under clindex. clindex is out everywhere now, so let's nuke it.
Shortens this file by 30%!
Reviewed By: quark-zju
Differential Revision: D10414815
fbshipit-source-id: 795c625afd128d94f09d0f7c1b294a7f7d8eea14
Summary: tweakdefaults was overriding two configs to reconcile core/fb behavior. Let's just fold it in to the default.
Reviewed By: quark-zju
Differential Revision: D10432786
fbshipit-source-id: 2f56baa2d11d7814bffc489e56cb155548d1fc97
Summary:
Manifestlog needs to use `00manifesttree.i` for filecache if treeonly is turned
on.
Teach `filecache` to also cache non-existed files and add `00manifesttree.i`
to `repo.manifestlog` check list.
Assuming inode cannot be 0, `cacheable` is a constant `True`. Drop code
dealing with `cacheable = False` accordingly.
Reviewed By: DurhamG
Differential Revision: D10417795
fbshipit-source-id: c8aedd36dc39592c86847bf4327ed9c46736bab0
Summary:
Aliases can be additionally configured with documentation by adding an entry of
the form:
```
[alias]
aliasname:doc = help text
```
The help text for the alias will be displayed if the user runs `hg help
aliasname`, and in the list of commands when the user runs `hg help commands`.
Reviewed By: phillco
Differential Revision: D10428399
fbshipit-source-id: 14338902fce6efab4c8d3b2541cdcf5b15385784
Summary:
Reorganise the top-level help commands to more useful categories, allowing them
to be listed in a custom order.
Reviewed By: phillco
Differential Revision: D10423184
fbshipit-source-id: 17f02ae201493397a448d108e781eca28a7b1d44
Summary: After D10407265 this is no longer needed.
Reviewed By: quark-zju
Differential Revision: D10407826
fbshipit-source-id: 5291981637d940a6d19cda1b814b81b745d9ca92
Summary:
This message as a warning is not actionable. wez, philco, markbt and
mitrandir77 seem to agree that this is a good approach.
Note that this message is still printed in debug mode.
Reviewed By: farnz
Differential Revision: D10336756
fbshipit-source-id: ea55d298569787d2e19eb1ffcfe6ff75935fb650
Summary: As a signal that sparse is cleaned up.
Reviewed By: phillco
Differential Revision: D10335537
fbshipit-source-id: 773a12a00cfe040f08871bf14423faa5eae9bdad
Summary: No point in tweaking it externally if we can just add relevant stuff.
Reviewed By: phillco
Differential Revision: D10335080
fbshipit-source-id: 47c009bf820f8f41f586b714181e82bea88dfa49
Summary:
Note that this is more aggressive than the tweakdefaults version; it doesn't allow named branches to be created or listed at all with the config set. As we want to delete named branches entirely, this seems reasonable.
I ported the `--new` flag, but made it optional, to make it easier to support both the core and tweakdefaults callers.
Saurabh Singh and I chatted with the releng team (Craig).
**tldr:** we can remove named branches from the UI but should keep the innards for now
Today the releng team doesn't make any branches (#continuous), but in the near-term future, they might either:
- create remote bookmarks in hg (similar to fbsource)
- create branches in SVN (which are then synced as hg branches by hgsubversion)
- We'll leave all the hgsubversion code to do this alone, just remove user-facing entry points
- They'll also need to be able to query for a commit's branch (`hg log -T '{branch}'` will still work)
- Once www is on hg, this possibility goes away and we can kill named branches for good
Reviewed By: ikostia
Differential Revision: D10401485
fbshipit-source-id: 6f2f3ae28af249bae1fdf782eb14fe7bfc472bb7
Summary:
Alas, it has to be config-gated for now, because a number of core tests depended on the empty-destination update behavior (which is sophisticated).
I'll send a followup diff to remove them.
Reviewed By: kulshrax
Differential Revision: D10376546
fbshipit-source-id: 2ac79e03a91551f96cf8e743c0ee9719557c4fbe
Summary:
The new hg absorb UX still prompts if there is nothing to be absorbed.
Instead, detect this condition and just exit with a message.
Reviewed By: ikostia
Differential Revision: D10382216
fbshipit-source-id: 5cd1aa9de457cc3475bcda064e8761b3ddf1ad40
Summary:
This greatly simplifies the implementation, too.
*updatetotally() change:*
`hg up` now passes a brev of `None` if the `--inactive` flag was passed. This avoids activating the bookmark if one was passed. However, it still needs to deactivate the active bookmark, if there was one. After looking at the existing code, I think it was just wrong.
The rest of this is just cleanup.
Reviewed By: markbt
Differential Revision: D10376544
fbshipit-source-id: e5ad8aa01acab906db4d3fc09c6450e3c48b59fb
Summary: Make a few help messages clearer.
Reviewed By: phillco
Differential Revision: D10356915
fbshipit-source-id: 277d4cecbd17b647d6dd01209ff6f93a926d37d4
Summary:
Replace the default help for Mercurial with a curated list of interesting
commands, categorized by their use case.
Reviewed By: phillco
Differential Revision: D10356916
fbshipit-source-id: 65e578a4bfde7b0ad04e7107f4e77d8ea882d78a
Summary: NB: `hg tags` still prints out all tags, just with a warning first. Maybe we could disable that entirely too while we're here.
Reviewed By: ikostia
Differential Revision: D10314175
fbshipit-source-id: 603a4dc46e05337189934803a18eef1188a02272
Summary:
It turns out core already implemented this, so let's just use their version.
Switch the facebook.rc config to it.
Reviewed By: ikostia
Differential Revision: D10306234
fbshipit-source-id: b0ee5e865ec4e0d8412dbd310a7e5017399b48c9
Summary:
Let's start moving tweakdefaults things into core.
Since we set the config in our rc files, I've renamed it.
Reviewed By: ikostia
Differential Revision: D10306236
fbshipit-source-id: 9a6b3bf7e22d8faf7b53446021660bb0923cf666
Summary: This tests works with `run-tests.py` but fails with buck.
Reviewed By: mitrandir77
Differential Revision: D10378486
fbshipit-source-id: 7112560a2d6b05b0a39f3df423ccac52bdc5202c
Summary:
Update hg absorb to use the templater and display the first line of the
description of the affected commits.
Also change the default behaviour to display the changes and prompt the user to
confirm that absorb is going to do what they want. Use the new `-a` option to
get the old behaviour.
Reviewed By: quark-zju
Differential Revision: D10366510
fbshipit-source-id: 23df2e5ab2d21a0805a16e7118686a29634d2918
Summary:
There are some code paths (ex. "purge --all") that sets watchman clock to None
unexpectedly. That makes the next command slow. Bypass it by detecting the case
and avoid setting clock to None.
Reviewed By: DurhamG
Differential Revision: D10188143
fbshipit-source-id: 4c35dfcee658ca2075d686da7d886e4b3a352b5a
Summary:
With treestate, watchman clock is not expected to be reset. But `purge --all`
bypasses some fsmonitor code paths and manages to reset the clock. Add a test
to show that.
Reviewed By: DurhamG
Differential Revision: D10188142
fbshipit-source-id: 4524979fb96d41fab881b7ee44ad9fa854081b26
Summary:
This extension exposes only the `record` command which can be easily
moved to core. This commit achieves the same.
Reviewed By: ikostia
Differential Revision: D10360759
fbshipit-source-id: 25f0c46aa3fa9b19ab8ba03a6b4e8598bc003c7a
Summary:
dirstate.untrack is used in two cases. Let's move the case where the
file is known to be actually deleted from disk to its own function. This allows
the treestate implementation to actually remove the entry from the structure,
which solves some case sensitivity issues.
Reviewed By: quark-zju
Differential Revision: D10317737
fbshipit-source-id: 09bc35d01cf5e52fdcbf45ea39c5d2f85fa39949
Summary:
There's a treestate bug on case insensitive filesystems where updating
across a case rename can cause junk to show up in hg status. A future diff fixes
this, but let's add a test first.
Reviewed By: ikostia
Differential Revision: D10317739
fbshipit-source-id: c1aa274cb55dd64cdb54ceadbd3c306151bd84d6
Summary:
dirstate.drop() is used in two different situations. 1. To make a
tracked file become untracked, and 2. To remove a file from the dirstate
entirely. In the treestate case, this difference matters, so let's split drop
into two functions, one to mark something as untracked, and the other to remove
it entirely.
In this first patch we just rename drop to untrack. The next patch will
introduce the new delete function.
Reviewed By: ikostia
Differential Revision: D10317738
fbshipit-source-id: 85950ab1b1a10cd481edcfbba6da445b3dbf6397
Summary:
The extension only offers one command i.e. `show` which can move into
core.
Reviewed By: ikostia
Differential Revision: D10302192
fbshipit-source-id: 9473ec8c80e52506e1b7de62b2c90a51c29419c1
Summary:
The functionality we care about is provided by the `fbshow` extension.
Therefore, lets replace the `show` extension with the `fbshow` extension.
Reviewed By: ikostia
Differential Revision: D10302193
fbshipit-source-id: e6d137f52427d5e6f124714d06b5b55ef0db7c3c
Summary:
This commit just removes the `show` extension.
This commit was split out of D10302193 to address the review comment in that
diff.
Reviewed By: ikostia
Differential Revision: D10359525
fbshipit-source-id: b77a08a7efbaa56eb5a4ceb2dbb3fecaba35f58b
Summary: Traditional shelve did not require `obsstore` to be enabled, which means that `saved bundle` messages were printed a lot. This change fixes all tests that expect these messages.
Reviewed By: singhsrb
Differential Revision: D10261258
fbshipit-source-id: 2b4b10ee94ae9b5723cb69fc8f7c29bc65613774
Summary:
The bug:
This happens because `repo['']` resolves to a `wctx`. This might be undesirable
in its own right, but before we fix that, we can work it around here.
Reviewed By: singhsrb
Differential Revision: D10261261
fbshipit-source-id: 8cb0e96762ffba24030f9e6bae8b0d0c108e3938
Summary: This is a first step towards making `obshelve` a real `shelve`.
Reviewed By: singhsrb
Differential Revision: D10261262
fbshipit-source-id: 7dccfbd7958cdf674c33c8ead73dcd279cebade6
Summary:
The functionality we care about is provided by the `fbamend`
extension. Therefore, lets replace the `amend` extension with the `fbamend`
extension.
Reviewed By: farnz
Differential Revision: D10320739
fbshipit-source-id: 5700d39f488777fcc4033f60ce0a51cda15ef2ad
Summary:
The logic we care about has been moved to core so we can delete the
`inhibit` extension.
Reviewed By: markbt
Differential Revision: D10276448
fbshipit-source-id: 44cabe5d561344cf8f196127ce52491e4654b598
Summary:
Treestate is missing its pathcomplete fastpath. This makes tab completion for
filenames slow on large repositories.
The Rust implementation already supported the feature, it just needs to be
called from the debugpathcomplete commands.
Reviewed By: quark-zju
Differential Revision: D10259633
fbshipit-source-id: f918d55c5bcd21805615710c0eb6302f8ea4b15c
Summary:
Parts of the treedirstate implementation were left in the extension. Since
treestate is now in core, and the two are intertwined, treedirstate should be
in core, too.
In doing so:
- Change the garbage collection behaviour to match that of treestate.
- Use the treestate config options for configuring repacking and garbage
collection.
- Make more of the code common.
Reviewed By: quark-zju
Differential Revision: D10258265
fbshipit-source-id: 89e82bc7662a3d1251fa9886751897cfc46cd66a
Summary: D10244968 changed the parser output but missed this test case.
Reviewed By: quark-zju, singhsrb
Differential Revision: D10261202
fbshipit-source-id: f3b6a4429b792e413f285fb33dfd6b64934ec477
Summary:
On the server, once we enter treeonly mode we still want to be able to
strip trees from the revlog.
Reviewed By: quark-zju
Differential Revision: D10248895
fbshipit-source-id: 15faec870370d4c28f6a83058eb1bf19cb414d29
Summary:
There's a bug where strip in a treeonly server will not strip the tree
revlogs. Let's add a test so a later fix can show the impact.
Reviewed By: quark-zju
Differential Revision: D10248897
fbshipit-source-id: f45e5abc59eeb9398ba17dd927a5f8eb50bc51af
Summary:
As we transition to treeonly, let's be sure every push contains a
manifest for every commit. To do so, let's add a hook that checks this.
Reviewed By: quark-zju
Differential Revision: D10244912
fbshipit-source-id: 6c2023b1cba5663ed386888a2007a005bc02f688
Summary:
There's an issue now where if a client is treeonly but does not have
sendtrees enabled, it could push commits to the server without manifest data.
Let's add a test for this.
The next commit will introduce a hook to catch this case.
Reviewed By: quark-zju
Differential Revision: D10244913
fbshipit-source-id: 2d6c3081c7190831e0368b0dab472d14e893471f
Summary: pest was upgraded by me to version 2.x which introduced the concept of End Of Input in the parsin rules. Modify the test case to be aware of this.
Reviewed By: singhsrb
Differential Revision: D10255598
fbshipit-source-id: 31ecf56f5cf75a16c799a93aefe7df9a3837b902
Summary: Create pack files instead of loose files for local data. Now this feature is under the config flag.
Reviewed By: DurhamG
Differential Revision: D9700992
fbshipit-source-id: b1ee897ac6fd2fe208f54f999b230353aebaca5b
Summary:
The age extension provides a couple of revsets relating to commit ages. Move
these to core.
Differential Revision: D10239000
fbshipit-source-id: 6ebf3aa263b72e4bbbe25bd516f77b17145e6e7f
Summary:
Currently `hg bisect` asks to run test on all commits it picked as a result of bisection, even if the changes between last checked nodes and chosen one relate to the files, which are not in the sparse profile.
And so it updates to the one of such commits, shows that "0 files were updated" and asks to run tests.
In this diff I check next potential node (result of bisecting) if there are changes between last good commit/last bad commit and the node. And go further to the next bisection iteration if the node can be skipped (nothing was changed since last good commit in sparse profile's files, so we can assume that the node is good as well).
If there are changes on both sides: between bad and the node, good and the node, I ask to run tests.
Reviewed By: markbt
Differential Revision: D9629683
fbshipit-source-id: f8c5d82f7b44ac14a9190c846ee65d635e447af1
Summary:
Verify SHA256 before writing content to the local blob store. This avoids
issues when the remote store does not response with the correct content.
Differential Revision: D10212567
fbshipit-source-id: 5683058670487df53fb14653d83c27da11367e76
Summary:
Move the `predecessors` and `successors` revsets to core, and unify the
variants as a single revset.
Before:
* `predecessors` (provided by fbamend) only returned the immediate
predecessors.
* `successors` (provided by core) returned the given commit and all eventual
successors.
* `allpredecessors` and `allsuccessors` (provided by fbamend) returned all
eventual predecessors and successors, not including the original commit.
Now all these revsets work like the old core `successors`, and additionally
have `depth` and `startdepth` parameters which can be used to customize the
behaviour. This makes them analogous to `ancestors` and `descendants`.
To maintain compatibility with existing users of `allpredecessors` and
`allsuccessors`, these revsets default to a start depth of 1, so they
do not include the original commits.
Reviewed By: quark-zju
Differential Revision: D10028043
fbshipit-source-id: e006700ccf430b7c5d6bd9588253872bce9a18ed
Summary:
Previously the manifestrevlogstore kept a copy of the changelog, the
store, and the svfs from the initial repo object. It's possible that these will
change or be replaced over time, so it's not safe to cache them. Instead let's
just fetch them again each time we need it.
Reviewed By: quark-zju
Differential Revision: D10156425
fbshipit-source-id: ac1e310b51be0377ba9bea75aa1806d256cc5b87
Summary:
This flow has a bug, so let's introduce a test to expose the break. The
next diff will fix the bug.
Reviewed By: quark-zju, singhsrb
Differential Revision: D10156424
fbshipit-source-id: 51d1d8da6728ee930dc1440cd3e685d7c5b5df2a
Summary:
At the moment they are not recorded because setrecordingparams may not be set
before hooks are called.
Also hooks weren't recorded because pushrebasereplacements weren't correctly
set and KeyError was thrown. This diff fixes it
Reviewed By: quark-zju
Differential Revision: D10147933
fbshipit-source-id: ef33d3e4efa7f3c42e7fcc22ee5f6e54068bf1dc
Summary:
Let's log reponame as well so that we can tell requests to different repos
apart.
I named config value `common.reponame` so that we can start using it in different extensions that also need reponame (for example, hgsql and infinitepush)
Reviewed By: farnz
Differential Revision: D10127734
fbshipit-source-id: 031c7393dfce43bf47474ac0878c1e68ddabec50
Summary:
It's useful to know because we can track down slow requests and potentially fix
them on the client.
Reviewed By: farnz
Differential Revision: D10127735
fbshipit-source-id: d21088b50065f322b0c312f048f5828b3d96f99f
Summary:
Add the `sparse switchprofile` command, which lets users specify a set of
sparse profiles they would like to switch to. This is equivalent to `hg sparse
reset` followed by `hg sparse enableprofile`, but in a single step.
Reviewed By: farnz
Differential Revision: D10141938
fbshipit-source-id: 5a1e547d21e3246237fffb17f3b512da18fac804
Summary: Reformat all opted-in python code with version `18.9b0` of Black.
Reviewed By: ambv
Differential Revision: D10126605
fbshipit-source-id: 82af0d645dd411ce8ae6b8d239e151b3730cd789
Summary:
In D10023543 stackpush was added, and conflict check can happen earlier. In
that case failed pushrebase wasn't recorded. This diff fixes it.
Reviewed By: quark-zju
Differential Revision: D10119679
fbshipit-source-id: ef12e00a43375151f81a95eec2d9f02db0a9b12b
Summary: When stripping rev x, revlog revisions with linkrev >= x should be deleted.
Reviewed By: phillco
Differential Revision: D10108592
fbshipit-source-id: 2f9f5663327c4494bd7e836ab24ffc7e507530f4
Summary:
* Rename takelock to dbwritable
The word "lock" is unclear what lock (local or SQL) it is. Make it clear it's
all about database writes.
* Rename waitforlock to enforcepullfromdb
Again, unclear what lock it is. It's also unclear what it does. Rename to
"enforce pull from db" to make it obvious.
* Rename syncdb to pullfromdb
"sync" is unclear about what direction to sync. Use "pullfromdb" to make it
clear. The hook name is unchanged for compatibility.
Reviewed By: phillco
Differential Revision: D10108594
fbshipit-source-id: fff405e2df9e926f5db436ef74cb5a9aacaebdb4
Summary:
Delay updating the working copy until after the transaction that created the
obsshelve commit has completed. This means we won't update away from the
user's changes until after their work is safe, so if the update is interrupted
for any reason, the user will be able up do a clean update and then unshelve to
get their work back.
Reviewed By: liubov-dmitrieva
Differential Revision: D10102089
fbshipit-source-id: 5709d3915a6c458ba7cfb37ba5e0be5c6e8fcbb2
Summary:
If the `update` step is interrupted, then the transaction that created the
obsshelve commit is rolled back, but the update can't be rolled back, so the
files that have been shelved are lost.
Reviewed By: liubov-dmitrieva
Differential Revision: D10102088
fbshipit-source-id: f5bcac5c92069cc6ff3d1d9b7fb7ee507003d8eb
Summary:
This is for sandcastle and other automated tools.
They will not need to run extra commands like hg log after amend to learn new
hash.
Reviewed By: markbt
Differential Revision: D10101887
fbshipit-source-id: 7c9931776a03f4335bdfe0a19e7d569e3dc4c4ba
Summary:
In the hgsubversion tests, a develwarn from another component (which doesn't
have a `config` key) crashes with a `KeyError`. Change to `get` so that
it doesn't crash.
Reviewed By: quark-zju
Differential Revision: D9699155
fbshipit-source-id: c360d5fb5ab7daf7f609d41e0ddfb7456b022666
Summary:
These are no longer necessary. Callers should use the sharedvfs to access the
shared repo's .hg directory.
Reviewed By: quark-zju
Differential Revision: D9699163
fbshipit-source-id: 9b9cd584d721c174a7eab06f6abcedc3a943233b
Summary:
Update hgsubversion to consistently use repo.sharedvfs, rather than using the
vfs of the srcrepo.
Reviewed By: quark-zju
Differential Revision: D9699165
fbshipit-source-id: 1f4dacdd23ebc3baaa0d09b65e45d4ceeb067559
Summary: Update references in core mercurial to use repo.localvfs instead of repo.vfs.
Reviewed By: quark-zju
Differential Revision: D9699162
fbshipit-source-id: 0401677d2b0a1340e66cffb7ee907a0d93aa6717
Summary:
Update bookmarks to work correctly when shared. Rather than copying the
bookmarks file over after transaction handling, update the file in both
locations when bookmarks are shared.
Reviewed By: quark-zju
Differential Revision: D9699158
fbshipit-source-id: 2f75aaac364ffe02e59441ac0f39fb7d8e5d5d2e
Summary:
Add `HG_SHAREDPENDING` which contains the path to the shared primary repository,
similar to how `HG_PENDING` contains the path to the local repository.
Repositories that are not shared check whether either of these refer to the
local repository path. Repositories that are shared check whether the pending
directory matches their own path, or the shared-pending directory matches their
shared path, via the new `trysharedpending` function.
This fixes the asymmetry in shared repos where pending changes made in a shared
repo were not visible in the primary repo, even though they were visible the
other way around.
Reviewed By: quark-zju
Differential Revision: D9699164
fbshipit-source-id: 31bc5fb2df6e9b9468b6ef39aabf877045c2a011
Summary:
Split the `repo.vfs` object into two. When a repo is not shared, these are
both the `.hg` directory of the repo. When it is shared:
* `repo.localvfs` represents the `.hg` directory of the local repository.
* `repo.sharedvfs` represents the `.hg` directory of the shared primary
repository.
The old `vfs` is an alias for `localvfs`. In the future, access through
this name will be deprecated to force callers to think whether they want
the local or shared hg directory.
Reviewed By: quark-zju
Differential Revision: D9699160
fbshipit-source-id: 6600df855c59b6df13e919399192789a873231c6
Summary:
Previously, statprof uses a naive `time.sleep`, which cannot be interruptted
easily (considering Windows compatibility). Change it to `threading.Event.wait`
so it can end early without waiting for the full cycle of the sampling period.
Reviewed By: wez, kulshrax
Differential Revision: D10067234
fbshipit-source-id: 24e5fc0ab05491cb3e7ff34024402842ac3d7d44
Summary:
This gives us more information about how large the hgsql operation is.
As we're here, move `sqlwriteunlock` to an earlier place so the logging is outside
the sql lock. Also document `unbundle` a bit so it's clear `repo.transaction()` only
takes sql lock in `unbundle` context.
Reviewed By: phillco
Differential Revision: D10056307
fbshipit-source-id: 5d3361b4044e6fcf01e60409ef1ecb34da34ccac
Summary:
RocksDB and InnoDB are highly compatibile. There is no need to test RocksDB
engine for every hgsql related tests. Only use rocksdb for 2 of the tests.
Reviewed By: phillco
Differential Revision: D10055068
fbshipit-source-id: f9b7ef546fe7d457b0390e49014ebbe56d3c12c1
Summary:
Our hypothesis is that if the prepushrebase hooks take a significant amount of time, the repo state will get out date, causing hgsql to degrade under load.
This uses the previous test to simulate new commits coming in to the database while a single server is busy running prepushrebase hooks. The new code causes a second sync to occur just after running the hooks.
Reviewed By: quark-zju
Differential Revision: D9999683
fbshipit-source-id: 43d2390b476d090a66353555247c9a623386e75a
Summary:
This test case is a bit simpler given the problem -- there's no need for the holding push hook.
Instead, you just need a server that's missing a public commit. (Simulate an `hg strip` on one of the servers. `hg strip` isn't allowed, so create another repo with the extra public commit instead.)
Reviewed By: quark-zju
Differential Revision: D10043378
fbshipit-source-id: 532d8a2791abe5aaa6b6932747c7e0145202e8fe
Summary:
The fast path avoids recreating the bundlerepo in the critical section, which
is like a reliable 0.8s win.
See the docstring in stackpush.py for details. It does not replace all use cases
that the old code path supports. So the old path is preserved.
Since it's a drop-in replacement, make it the default.
Reviewed By: phillco
Differential Revision: D10023543
fbshipit-source-id: eaceb9ae5067ab9040aa10cc65170ae54abd3331
Summary:
The next patch will change pushrebase code path so it does not have to go
through pushrebase._commit. Change globalrevs to wrap the lower-level
repo.commitctx to make sure commits have global revs assigned. Pushrebase
and hgsubversion wrappers are removed accordingly.
This also affects "hg commit" running in the server-side repo. Therefore
the test is changed to keep the change minimal.
It's also incorrect to reset the next revision counter at `repo.invalidate`.
It should only be reset at transaction abort. The original concern was
to get an up-to-date view of the revision number. Doing it in `repo.invalidate`
is risky. Enforce hgsql lock when reading the number to make it safer.
As we're here, change the transaction code so it does not wrap `_abort`
unnecessarily for nested transactions.
Reviewed By: singhsrb
Differential Revision: D10023541
fbshipit-source-id: 82d4b57dc2eafa8bc3cdf553e891db6e8c5ff741
Summary:
Inside the critical section, "needsync" runs twice and each time it scans
O(bookmarks + heads), which could be slow.
If the repo is actually up-to-date, there could be a faster path for testing -
just hashing all bookmarks and tip. Note: "heads" is skipped to save some time.
Since commits are append-only, "heads" change implies "tip" movement. So
checking "tip" is enough.
As we're here, fix the empty repo case so syncing is skipped for an empty repo.
Since this is a drop-in replacement for `needsync()[0]`, turn it on by default.
Reviewed By: phillco
Differential Revision: D10022866
fbshipit-source-id: f9e304865db3575515d66444762f21071d5665a3
Summary:
When entering the critical section (creating a transaction), hgsql might wait
for SQL lock for a long time (minutes). And it currently does nothing during
that wait time. If the lock was acquired after a long time, the work of
catching up with what the database already has might be too much. That is a
waste within the precious hgsql lock held critical section.
Instead of waiting for the lock for N seconds and doing nothing, wait for k
(k < N) seconds and also try to sync with SQL periodically. This makes the
repo closer to what the database has and can reduce work needed to pull from
the database in the critical section.
This adds CPU pressure to the MySQL tier. I added a config option to control
it. Worse case, we can change `hgsql.syncinterval` to make the sync less
frequent.
As we're here, try document the configs.
Reviewed By: phillco
Differential Revision: D10002578
fbshipit-source-id: bd72d8225c919aa2bc62743de1e1d3f27cba606a
Summary: We want to add a test for this case before fixing it.
Reviewed By: quark-zju
Differential Revision: D10034448
fbshipit-source-id: 9244c57311d71a0bf1d75643caa31960a2f3519f
Summary: We need an in-depth test that tests both pushrebase and hgsql and shows how often we are reading uncached manifests. This is such a test.
Reviewed By: quark-zju
Differential Revision: D10023492
fbshipit-source-id: cf9e76e505c3bc478a7fc13d8d1adf1adeb6c1e4
Summary:
This allows versions that don't know about storerequirements still access newly
created repos with this version. We will turn this on at a later date.
Reviewed By: singhsrb
Differential Revision: D10033964
fbshipit-source-id: e1065e05c33544d0287eda5eb852baff07c13147
Summary:
I want to centralize knowledge about specific sqllocks so I can change the
locking mechanism in the next commits of this stack
Reviewed By: quark-zju
Differential Revision: D9993477
fbshipit-source-id: 9398476b0ba8c3175ce84a7e0a809bbb8e60b7df
Summary:
With hgsql.verbose turned on, print useful messages like how many commits the
server is syncing, how long the lock was held or waited, to stderr so they are
visible to the client. This is useful for the client to understand why the
server "hangs" for a couple of seconds, and is useful in tests.
As we're here, drop no-flake8 comment.
Enable the flag for the hgsql-sync test to verify it.
Reviewed By: phillco
Differential Revision: D10019937
fbshipit-source-id: 8d304ce5208dbc5b92ed20f69daba02e9040c73f
Summary:
Sometimes watchman can return a very long list of files. That would make
blackbox rotate too frequently. Truncate that list.
Reviewed By: phillco
Differential Revision: D9997658
fbshipit-source-id: 1314b7c299a5f50ed344e0b85befd4c0525b1da7
Summary:
Previously, the `[profiling]` config section defines the profiler. Now it can be
defined by other sections, `[profiling:1]`, `[profiling:2]`, etc. Those sections
are sorted by name, and the first one with `enabled = true` is effective.
This makes it possible to specify secondary profiling configs, like having a
low-overhead profiler to be always enabled and logged if the command exceeds
certain threshold:
[profiling:background]
enabled = 1
freq = 1
minelapsed = 30
output = blackbox
type = stat
The config will only be used when the primary profiling (`--profile`, or
`profiling.enabled`) is disabled.
Reviewed By: markbt
Differential Revision: D9828548
fbshipit-source-id: 301a2b049d569d6fc065d05ab5b02e85b4ea8cc5
Summary:
Make "profiling" a variable. No logic change.
The next patch changes the config section name dynamically.
Reviewed By: markbt
Differential Revision: D9828551
fbshipit-source-id: b6b76012462f2dc1a76cd162381d949a94dc498b
Summary:
This config allows us to only log the profiling result if it exceeds a given
threshold. It's helpful for always enabling profiling and only log meaningful
data.
Reviewed By: markbt
Differential Revision: D9828549
fbshipit-source-id: f7e8f7e1e1f834d980519c2a639f3c7fd02f7f42
Summary:
In pushrebase code path, `_getmanifest` needs to be used to obtain the manifest
correctly for pushes from treeonly repos. The merge commit code path missed this
change. Fix it.
Besides, the rename checking code patch can also hit flat manifest errors via
repo.commitctx -> ... -> fctx._copied -> ctx.filenode -> ctx._fileinfo. Patch
that path too.
Reviewed By: phillco
Differential Revision: D9977904
fbshipit-source-id: 8c51aa9cc6bd85b0147d88c61a5af074e36cdb61
Summary: Part of tech debt week; let's make it so any time we reject an update due to conflicts, we print what the conflicts are.
Reviewed By: quark-zju
Differential Revision: D9784659
fbshipit-source-id: 1ee232b728bb602dbeaa4d2ecbc11016edd48dce
Summary:
This commit adds a way to run the `globalrevs` extension only in read
only mode. This is useful in cases where repository `x` pulls from repository
`y`, both the repositories have `globalrevs` and `hgsql` enabled, we only
want to embed `globalrevs` in the commits for repository `y`, and we want to be
able to read the `globalrevs` in repository `x`.
Reviewed By: phillco, quark-zju
Differential Revision: D9978146
fbshipit-source-id: b9530e8e9cfadc28e2cd30e51649cee9e6e144bf
Summary:
The long filename encoding method introduced in D8527475 is insufficient for
files that have very long names that include some underscores.
For these files, we encode the underscores as `:`. This character is valid for
filenames on Linux, but won't work on Windows.
Reviewed By: mitrandir77
Differential Revision: D9967059
fbshipit-source-id: 67ea662f48f9fcc40e00b36c86697f3be6aac978
Summary:
Create Python bindings so that the BookmarkStore can be interacted with
from Python.
Reviewed By: quark-zju
Differential Revision: D9768565
fbshipit-source-id: 00d75b5250d8bc7dbeddd90d80ff4a23c60d00f9
Summary:
When syncing with the commit cloud, if we have commits that are hidden locally,
but visible in the cloud workspace, and no local obsmarkers waiting to be
synced, revive the cloud-visble commits.
This prevents two repos from oscillating between two views when they have
a conflicting set of obsolescence information that has not been cloud shared.
Reviewed By: liubov-dmitrieva
Differential Revision: D9845984
fbshipit-source-id: 2f44ff8dbc636e2afa56d5efbb3ea5114472c41c
Summary:
If two repositories have conflicting obsmarker information that has not been
shared through commit cloud, then they can enter a state where they fight over
the visibility of the commit, and the cloud workspace oscillates between two
views. This adds a test to demonstrate this occurring.
Reviewed By: quark-zju
Differential Revision: D9845983
fbshipit-source-id: d5c74bed25da5721f273e53df10bf7b88efe2850
Summary:
In the current implementation, the `globalrevs` extension complains if
the `hgsql` extension is enabled but bypassed. This creates issues in cases
where the `hgsql` extension is bypassed intentionally as in D5012225. This
commit changes the logic in `globalrevs` extension to not wrap any extensions
if the `hgsql` extension is bypassed intentionally instead of raising an
exception.
Reviewed By: quark-zju
Differential Revision: D9883231
fbshipit-source-id: c57eaad4f661b69752a6457e6ecdd43f377285c1
Summary:
Remove newdoc from test-check-code - it's a Sphinx project, so MiniRST
checks don't apply. Also remove the unused imports from the Sphinx config
file.
Reviewed By: farnz
Differential Revision: D9851834
fbshipit-source-id: c17e83fd96d39423eeedfb8a6cfae382f88c4d5f
