Commit Graph

43742 Commits

Author SHA1 Message Date
Durham Goode
4588cc18c8 mutabledatapack: write version number header
Summary:
The mutabledatapack format has a one byte header containing the version
number.

Reviewed By: quark-zju

Differential Revision: D8305653

fbshipit-source-id: c4a96dc48e64acd2c5849034e5d90b87363fbc8d
2018-07-05 14:53:15 -07:00
Durham Goode
99a11bbb24 mutabledatapack: use hash of contents as name
Summary:
Implements the logic that builds a hash of the contents of the pack
file and uses it as the name.

Reviewed By: quark-zju

Differential Revision: D8305654

fbshipit-source-id: d1270e7519a7718aa5427f3be5cdc0cd0dee2fe2
2018-07-05 14:53:14 -07:00
Durham Goode
3f467bd21f mutabledatapack: implement add()
Summary:
This is the start of a rust mutable datapack implementation. The first
diff adds a simple add function. Later diffs will add the logic that builds the
index, serializes the index, and computes the final hash name.

Reviewed By: quark-zju

Differential Revision: D8304036

fbshipit-source-id: db05c2b845e51a3552c039b7fc0b8f4cc0ff0852
2018-07-05 14:53:14 -07:00
Durham Goode
8057817dc1 revisionstore: add read/write functions to Metadata
Summary:
In a future diff we'll be serializing and deserializing metadata in
datapacks. Let's add the reader and writer functions for Metadata and some unit
tests.

Reviewed By: quark-zju

Differential Revision: D8303603

fbshipit-source-id: 7e7a7aa218c05179b205abf8b151b1488be674b3
2018-07-05 14:53:14 -07:00
Matt Glazar
9b7a4d32a1 templater: allow non-ASCII fillchar in pad()
Summary:
Hg's `pad()` template function can only fill with a single character. To assert this, Hg checks the length of the fill string. Because the fill string is a Python 2 `str`, Hg rejects UTF-8 `'☃'` (`'\xe2\x98\x83'`) because its length is 3.

Instead of checking the length of the fill string, check the width of the fill string while taking encoding into account. This lets us use ☃ as a pad fill character.

For ASCII fill strings in `pad()`, this diff should not change behaviour.

Reviewed By: phillco

Differential Revision: D8730727

fbshipit-source-id: ad015aa4cae274160bc2726e0cb5e570e6edcc1e
2018-07-05 14:20:44 -07:00
Liubov Dmitrieva
8eba8372c3 commitcloud: cloud sync always wait for the background process to complete
Summary: before the waiting provide all the information to the user

Reviewed By: markbt

Differential Revision: D8734860

fbshipit-source-id: 9c0203b0879091d1d367c2ded9eccd8db4ee1bb9
2018-07-05 11:35:16 -07:00
Jun Wu
e4f29da588 hardlink: whitelist apfs
Summary: `hg debugfs` can report `apfs` correctly on latest mac. So let's add it to whitelist.

Reviewed By: ikostia

Differential Revision: D8725764

fbshipit-source-id: a9eb0db9d1f75cdd64c28b5c2f41451955758e18
2018-07-05 10:06:41 -07:00
Mark Thomas
e96dc290b9 tests: relax output for rename commit in treeonly-copyamend test
Summary:
The output of the rename commit can vary depending on how many files hg needs
to download as part of the rename.  This makes this test flaky when run with
watchman in the stress runs.

We don't actually care about the commit output here, so just match anything.

Reviewed By: mitrandir77

Differential Revision: D8731713

fbshipit-source-id: c2eded9115d2348c11fcb79eb425b923d073eddb
2018-07-05 09:50:59 -07:00
Phil Cohen
b32405f449 rebase: replace opts['noconflict;'] with opts.get('noconflict')
Summary: simpkins found a user getting a KeyError with the former.

Reviewed By: simpkins

Differential Revision: D8730525

fbshipit-source-id: 40d80f17e0ab8b2905fc53377e501546c21feaae
2018-07-03 17:49:44 -07:00
Mark Thomas
2ca405cbe0 remotefilelog: add config option to disable descendantrev fastpath
Summary:
The fastpath for adjusting linknodes using the fact that a filenode ctx was
created from a descendant rev can sometimes cause lots of treefetches.  Add an
option to disable it.

Reviewed By: mitrandir77

Differential Revision: D8691202

fbshipit-source-id: f196565929a2cf9b850c1c7c35f3ba6fa90f9031
2018-07-03 12:55:46 -07:00
Mark Thomas
af37374ca6 remotefilelog, treemanifest: add test demonstrating excessive gettreepacks on amend
Summary:
When amending a commit containing a file rename for a file with a long history
where the original content was usually on a separate branch, we end up in
`remotefilelogctx._adjustlinknode` for every commit the file was modified in.
Each of these looks up the manifest for the file, which causes a tree fetch.

Reviewed By: quark-zju

Differential Revision: D8691179

fbshipit-source-id: 088a6b402a79d37790ce7ac4481a39c657f690ee
2018-07-03 12:55:44 -07:00
Phil Cohen
0cc316cdc4 tests: run check-code.py in parallel via xargs (2x-4x speedup)
Summary:
This cuts the total time for this test from:

Laptop: 24s -> 12.4s
Devserver: 23s -> 5.8s

Reviewed By: quark-zju

Differential Revision: D8721177

fbshipit-source-id: f2922093a72c29f05d08967a77d83fbc65c0ed14
2018-07-03 11:36:17 -07:00
Jun Wu
c7e304d4e1 dirstate: fix symlink handling about unknown files
Summary:
Fix an issue where pathaudit failed (see [1] for what pathaudit does) files
showing up in "hg status" with fsmonitor enabled.

This is a problem with all dirstate implementation. Treestate makes it
easier exposed, as it merges "notefiles" with "nonnormalset", instead of
replacing an exisiting "notefiles" set.

[1]: https://www.mercurial-scm.org/repo/hg/rev/2cbd27f4f3c4

Reviewed By: DurhamG

Differential Revision: D8721020

fbshipit-source-id: 7c6c8c2f202c0da4c3eeee3c9b1ce10bf7970dd8
2018-07-03 05:50:52 -07:00
Jun Wu
a397871bd0 dirstate: add a test showing a symlink problem
Summary:
Files with one of its directory being a symlink should be considered as
outside the repo. The test demonstrates that with fsmonitor, all dirstate
implementations have problems showing auditfail [1] files.

[1]: https://www.mercurial-scm.org/repo/hg/rev/2cbd27f4f3c4

Reviewed By: DurhamG

Differential Revision: D8721021

fbshipit-source-id: fcb9808b637993c4d0c173d771fd7220afb52d83
2018-07-03 05:50:52 -07:00
Phil Cohen
cd0765f073 tests: make check-code.py output in a sortable way, and sort it
Summary:
Print the output from check-code.py in a way amenable to sorting (mostly, print each offense on one line instead of several)

The next diff will explain why.

Reviewed By: quark-zju

Differential Revision: D8721178

fbshipit-source-id: d814f94c151c6251f827ed6fe099e170e0b5353a
2018-07-02 23:50:57 -07:00
Phil Cohen
b879f0f5b3 tests: fix legitimate test-check-code.t failures
Summary: Mostly os.getcwd() or os.environ issues.

Reviewed By: DurhamG, quark-zju

Differential Revision: D8721169

fbshipit-source-id: 6615382775ce29dff814d830841f0c7eb4cad901
2018-07-02 23:50:56 -07:00
Phil Cohen
1064b3d79c tests: remove check-code complaint about grep's context flags
Summary:
It blames to here: https://www.mercurial-scm.org/repo/hg-committed/rev/0c09afdf5704

I'm pretty sure we don't care about Solaris :)

Reviewed By: DurhamG

Differential Revision: D8721170

fbshipit-source-id: 0c500ccb28828862136f6fcc44181477f5a21eee
2018-07-02 19:21:01 -07:00
Phil Cohen
b5f65c2cff tests: move "no new files in root" check to its own file
Summary:
This takes about 4.5s on my laptop. By moving it to its own file, we can run it in parallel and save from the total check-code time.

Laptop: 31.74s -> 25.53s
Devserver: 24.5s -> 22.21s

Reviewed By: quark-zju

Differential Revision: D8721156

fbshipit-source-id: 6d4d2c86517239f1762a413bc5d3dfefe184eaca
2018-07-02 19:21:01 -07:00
Phil Cohen
b59890687a overlayworkingcontext: properly handle copy-on-clean context
Summary:
This is a more minimal fix compared to D8676355, to handle extra calls to `markcopied` from `copies.duplicatecopies`.

Let's just make overlayworkingcontext properly support receiving copy information on (non-deleted) files, which it should do anyway.

This leaves the incorrect non-IMM behavior, but since it just involves incorrectly creating an empty commit, that's probably fine.

Reviewed By: quark-zju

Differential Revision: D8697039

fbshipit-source-id: 0c204c47e6b5031eb20928c1844be3aebcb96edb
2018-07-02 15:50:18 -07:00
Jun Wu
f526d1a4ed treestate: update clock when watchman returns is_fresh_instance=True
Summary:
When watchman returns is_fresh_instance=True, it means the old clock value
cannot help watchman to do less work. Therefore the clock value needs to be
updated to make the next status call faster.

Reviewed By: phillco

Differential Revision: D8714863

fbshipit-source-id: 1eb2bf876b21bd0f17134a3b93b06b759746fd4c
2018-07-02 13:35:42 -07:00
Phil Cohen
47c81d5606 tests: fix the build (v2)
Summary: eyes

Reviewed By: kulshrax

Differential Revision: D8716471

fbshipit-source-id: 7b2b29cc845cc03af2387190482ce356337ff451
2018-07-02 13:05:35 -07:00
Phil Cohen
df568f4e22 tests: fix the build
Summary: eyes

Reviewed By: singhsrb

Differential Revision: D8716279

fbshipit-source-id: aa214a554455513d5f121560b0c69278e5f8e982
2018-07-02 12:35:48 -07:00
Phil Cohen
05ac4a77f5 tests: modernize test-fb-hgext-fbamend-restack.t
Summary:
This test is 990+ lines and takes 1+ minutes to run on my laptop.

- Use `newrepo` instead of a custom reset function
- Use `enable` and `setconfig`
- Split out the auto-restack part into its own function

Reviewed By: quark-zju

Differential Revision: D8706316

fbshipit-source-id: fb48208e3225b376a0e4d06021ef82e8c4255d08
2018-07-02 11:36:46 -07:00
Phil Cohen
0be0108bed rebase: replace --tool :abort with rebase --noconflict
Summary:
I was thinking about what we need to enable automatic restacking on non-conflicting amends (see the next diff), but I realized the --tool :abort tool I implemented in D8493065 doesn't really work well for this for three reasons:

- It can't handle the case of mergedriver having to run, which is the other reason we'd have to break out of IMM.
- It hard-codes the merge tool. The user might want to specify another tool that doesn't recreate conflicts or solves them differently.
- It'd force callers to detect if IMM will be used (to prevent an on-disk merge hapening by mistake). A flag can implement this much more easily.
- As I learned when writing D8493065, it would require hardcoded logic in every command anyway to support the non-IMM case. (Just raising an abort will leave most commands with an interrupted state.)

So, I think we should replace it with this flag to make auto-restacking work very reliably (we could add it to graft, too). If there was some big demand for --tool :abort we could always add it back in the future.

Reviewed By: quark-zju

Differential Revision: D8701897

fbshipit-source-id: ea3b92d0a224a8ce43edb120b53bec241d92a61d
2018-07-02 11:36:46 -07:00
Phil Cohen
46358b47c2 merge: add the type of conflict (file or driver) and path(s) to InMemoryMergeConflictsError
Summary: This will let us print more helpful/precise error messages.

Reviewed By: singhsrb

Differential Revision: D8699551

fbshipit-source-id: 060058eff925d94ffa8e6b4432c4e51bd93b2945
2018-07-02 11:36:46 -07:00
Phil Cohen
200ae35c45 rebase: remove experimental.inmemorydisallowedpaths
Summary: This config was needed when IMM restarts were long and painful, but now that they're not, this clunky and inaccurate hueristic can be removed, which simplifies the control flow a bit.

Reviewed By: singhsrb

Differential Revision: D8700868

fbshipit-source-id: 0aeac26e5c41f7f0260ee826f05a8f9047dbad5a
2018-07-02 11:36:44 -07:00
Phil Cohen
7350e2a60c rebase: remove rebase.experimental.nomergedriver
Summary: This configuration is set everywhere, and the spectre of stateful preprocess() functions was a Facebook-only abomination that hopefully won't reoccur. Delete this config.

Reviewed By: quark-zju

Differential Revision: D8700125

fbshipit-source-id: b5ef163f988b7bb79310275d54afea3eab2c23a2
2018-07-02 10:35:25 -07:00
Phil Cohen
2e51a83397 rebase: remove old switching behavior
Summary: The new logic is on at 100% without problems, so axe the old.

Reviewed By: singhsrb

Differential Revision: D8698417

fbshipit-source-id: 0126c61c7bbfa49609354415f7cc61e861c1f2d5
2018-07-02 10:35:25 -07:00
Liubov Dmitrieva
14ff03f833 fbclone: enable commit cloud check
Summary: enable call of hg cloud rejoin in fbclone

Reviewed By: markbt

Differential Revision: D8677927

fbshipit-source-id: e50ad2dd5db838c4051061da46fa067153eb866d
2018-07-02 03:04:54 -07:00
Durham Goode
7add7dc1ef treemanifest: use manifestlog's mutable pack for prefetches
Summary:
The manifestlog has some mutable packs that it persists and manages the
lifetime of. As long as every tree writer uses these, we'll only get one new
pack per process/transaction.

The prefetch code was not using these though, so things like 'hg log -p' would
produce many packs over the course of many fetches. This change fixes it to
always write to the same pack.

Reviewed By: phillco

Differential Revision: D8670684

fbshipit-source-id: e56590c0ced84a99a06a94028d6a4e14bcc22ab6
2018-06-29 15:51:40 -07:00
Durham Goode
0bd796a5f6 clone: close repo during clone
Summary:
A future diff will make us depend on repo.close() being called to
finalize the writing of some tree data. It turns out we didn't call close on the
destination repo during a clone, which caused some tests to change. Let's call
close on that repo to prevent a test change in the next diff.

Reviewed By: phillco

Differential Revision: D8670680

fbshipit-source-id: ad63ccfc51ec4c3de2dad494fdb1ae8f6ba8e40a
2018-06-29 15:51:39 -07:00
Durham Goode
742de6048e tests: update p2p test to use different caches for different peers
Summary:
An upcoming diff changes the way we use the hgcache and this test is
mildly affected because it has two local peers using the same cache. Let's
change the test so the two peers use different caches, that way the test
requires no changes for the next diff.

Reviewed By: phillco

Differential Revision: D8670681

fbshipit-source-id: acd0051aa07974fa6dc2850f69957ff9ae701c83
2018-06-29 15:51:39 -07:00
Durham Goode
88c9fd7935 mutablepacks: add mutable shared packs
Summary:
We already had mutable local packs, which were used to allow all local
writes to go to the same pack. Let's add an equivalent mutable shared pack. A
future diff will change shared writes to go to these packs.

Reviewed By: phillco

Differential Revision: D8670682

fbshipit-source-id: cbe3ab40b30f5ff53ece23914a95ffeba8b03ec3
2018-06-29 15:51:38 -07:00
Durham Goode
6dffc619da mutablepacks: refactor mutable packs to be local only
Summary:
ManifestLog._mutabledata/historypack is used to persist a single set of
mutable packs for all local writes in the process. In a future diff we'll be
addeding the same concept for shared packs as well, so let's refactor the
current logic to explicitly be named _mutablelocalpacks.

Reviewed By: quark-zju

Differential Revision: D8670679

fbshipit-source-id: 8b1a04490c71525eca9466a7288dd7fd23991089
2018-06-29 15:51:38 -07:00
Durham Goode
a444e85875 record: remove name encoded backups
Summary:
Previously we were encoding the full filepath into a single filename to
use as a backup file. If the full path was over 255 characters, then we couldn't
create a file with the name.

Since the user never sees this name unless they run --debug, let's just get rid
of the encoded name and use a purely temporary name.

Reviewed By: farnz

Differential Revision: D8695749

fbshipit-source-id: 18b6d00e5f4c856b7dd6073e9d8c9677c4baea69
2018-06-29 15:35:59 -07:00
Saurabh Singh
a468f0d5b0 test-global-revs: add test for simultaneous pushes
Summary: See the commit content itself for the details on the testing.

Reviewed By: quark-zju

Differential Revision: D8695360

fbshipit-source-id: c7d4bb85a1c5ec3648a00e171b3ad592066d5daf
2018-06-29 15:05:50 -07:00
Liubov Dmitrieva
e22322c2e5 commit cloud subscriber: skip cloud sync if can't resolve interngraph host
Summary:
this will reduce cloud sync errors and unnecessary cloud sync calls

the daemon triggers cloud sync on service start/restart
it is not always the time when the machine online (and connected to correct network), so we get cloud sync errors

Reviewed By: markbt

Differential Revision: D8692972

fbshipit-source-id: 59033fd4c3e7c30100d82b908442bbf1ebea9322
2018-06-29 12:20:11 -07:00
Jun Wu
edf7c6cfe3 drawdag: support removing files
Summary:
Previously, drawdag can only create commits which add or modify files.
In some cases, removing files is needed. This diff makes it support removing
files by special comments like "(removed)" and "(renamed from ...)".

Reviewed By: phillco

Differential Revision: D8698324

fbshipit-source-id: 97da946332fe64275cef40ba9ba6ac60846d7de9
2018-06-29 11:35:29 -07:00
Saurabh Singh
938a9b048d test-global-revs: introduce tests for the globalrevs extension
Summary:
This commits just adds some tests for the `globalrevs` extension. See
the commit content itself for the details on the testing.

Reviewed By: DurhamG

Differential Revision: D8364382

fbshipit-source-id: f32b9fedd70f5f8ad40a034ce0d28867f9d1bdb4
2018-06-29 11:08:15 -07:00
Phil Cohen
73acd830dd mergedriver: fix the invalidation flakiness by setting sys.dont_write_bytecode
Summary:
D8464848 added code to invalidate merge drivers after running them, for reasons described in that diff.

But it turns out deleting from sys.modules is not enough, because Python will occasionally write a `.pyc` of the original hook, and re-use it when loading again, thus bypassing the invalidation.

This doesn't happen at all on my Mac, but happens approximately every other time when running on CentOS. Thanks to DurhamG who discovered the flakiness.

This fixes the flakiness of the test, and AFAIK this would be a problem in the wild, too, though we haven't heard of it yet.

Reviewed By: singhsrb

Differential Revision: D8675720

fbshipit-source-id: bec8fec6af9a362db1f0dd5d262932cbda6137ff
2018-06-29 10:47:38 -07:00
Simon Farnsworth
03a6aad6f7 ui: remove machine-generated blocked tags
Summary: We initially split out lots of different causes of slowness into their own unique tags, so that we could identify the source of slowness. Now that we've identified all the big causes, wrap them all back up into summary categories - we can change back to detailed tags at callsites if we want to separate out phases.

Reviewed By: ikostia

Differential Revision: D8664314

fbshipit-source-id: 6e141382479ca044847db1611f885e08f24f986b
2018-06-29 05:34:00 -07:00
Jun Wu
82298d423e fsmonitor: avoid cyclic reference in reposetup
Summary: Break the cycle by using local variable.

Reviewed By: chadaustin

Differential Revision: D8686397

fbshipit-source-id: e1846d33472066466814ab2c0c666f72b8393ff3
2018-06-28 17:35:21 -07:00
Jun Wu
4ba555977c vendoredcrates: upgrade zstd-sys to the latest
Summary:
zstd has dropped `ZSTD_TARGETLENGTH_MIN` [1]. Let's upgrade our code to be
compatible.

[1]: c2c47e24e0

Reviewed By: DurhamG

Differential Revision: D8683180

fbshipit-source-id: 66cbab1ddd254b1e0b91232565b4d512810ba03d
2018-06-28 15:08:01 -07:00
Jun Wu
f45557f483 commitcloud: avoid cyclic reference in reposetup
Summary:
A reference of `repo` is kept in the `finalize` closure. That forms a cycle:

 - finalize needs repo
 - repo.transaction needs finalize

Break the cycle by making `finalize` use `self` passed to `repo.transaction`
instead.

Reviewed By: chadaustin

Differential Revision: D8676286

fbshipit-source-id: e27bed996883e2bf22cdeec41fcf1ab3e771b208
2018-06-28 14:20:41 -07:00
Jun Wu
bb9e6e0a18 fbsparse: avoid cyclic reference in reposetup
Summary: When wrapping the repo object, use local variable if possible.

Reviewed By: chadaustin

Differential Revision: D8676284

fbshipit-source-id: d35d9b7b9357a8e16d4eb10032911ecbdc34b907
2018-06-28 14:20:41 -07:00
Jun Wu
d17c133cd7 remotefiellog: avoid cyclic reference in reposetup
Summary: When wrapping the repo object, use local variable if possible.

Reviewed By: chadaustin

Differential Revision: D8676283

fbshipit-source-id: 627e5fa4cc126efff42f29cf463fa90b97d9a0e0
2018-06-28 14:20:41 -07:00
Jun Wu
9aca1a1da1 fastmanifest: drop global uiproxy
Summary:
fastmanifest uses a `uiproxy` semi-global variable to workaround a chg issue
where `uisetup` received an out-of-date `ui` object.

That's no longer necessary D7840237. Removing the `uiproxy` global variable
fixes the memory leak.

Reviewed By: chadaustin

Differential Revision: D8676285

fbshipit-source-id: a5e042e4f61460ce53d43a231f07d70f27a52f19
2018-06-28 14:20:41 -07:00
Jun Wu
e4ff05e792 treemanifest: avoid cyclic reference in reposetup
Summary: Replace references to non-local variables with local variables.

Reviewed By: chadaustin

Differential Revision: D8675979

fbshipit-source-id: 8c35db5f3ea8a7ee79dad163928341607b37bb1c
2018-06-28 14:20:41 -07:00
Jun Wu
29bed9173c blackbox: use weakref to avoid cyclic reference
Summary:
The blackbox extension uses a global variable for `lastui` in case certain ui
objects do not have an associated repo. That `lastui` reference makes the `ui`
(and `repo`) harder to be dropped. It also sets `ui.repo` which could form a
cycle if `repo.ui` is that `ui`.  Fix it by using weakref.

Reviewed By: chadaustin

Differential Revision: D8675908

fbshipit-source-id: 7d0c083cc5c7a3ac134379732de1e0c408a1d5fc
2018-06-28 14:20:40 -07:00
Jun Wu
b22b529f1b hgsubversion: avoid cyclic reference in reposetup
Summary:
Continue with the sampling fix. hgsubversion is the second trouble marker.
It uses similar patterns (reference to non-local variables in closure).
Fix it by using local variables.

Reviewed By: chadaustin

Differential Revision: D8675439

fbshipit-source-id: bff7a827608fbf4a2662e6e9956ddb733c6a3bb5
2018-06-28 14:20:40 -07:00