Commit Graph

43664 Commits

Author SHA1 Message Date
Jun Wu
c43312ad9c indexedlog: utils: move xxhash to utils
Summary: The function will be reused in another module.

Reviewed By: DurhamG

Differential Revision: D8156522

fbshipit-source-id: 2aff6f2e4b8fc9b5d2c000e12ac2d940f7fab407
2018-06-05 00:12:29 -07:00
Rain ⁣
10aea2f2d1 improve help text for split
Summary:
per https://fb.facebook.com/groups/scm/permalink/1672273556155591/

(Note: this ignores all push blocking failures!)

Reviewed By: quark-zju

Differential Revision: D8266343

fbshipit-source-id: a48e3f95c061cb8e6031f5844c49139602843238
2018-06-04 21:35:25 -07:00
Durham Goode
ecb3759c4a treemanifest: fix tree conversion fast path usage
Summary:
There was a bug where if we tried to convert a flat manifest into a
tree while the flat manifest commit was being committed, the fast path
conversion would produce incorrect results because it tried to access the commit
data.  If len(changelog) == 5, then changelog.revision(5) returns the nullid
data, which caused the bug.

This is absurb behavior from the changelog, so I will change that in a future
diff.

Reviewed By: phillco, singhsrb

Differential Revision: D8270372

fbshipit-source-id: 27bbfb2e54de35553b880954f06e76fe4fc0d47b
2018-06-04 18:23:03 -07:00
Durham Goode
1ac8657f83 treemanifest: require setting the start path for SubTreeIterator
Summary:
In a future diff we'll be adding the ability to iterate over just a
subsection of a tree. In order for the resulting paths to be correct, we need to
be able to prime the start path. To do so, let's add a path argument to the
SubTreeIterator, and set it to empty string everywhere it's currently used.

Reviewed By: quark-zju

Differential Revision: D8216727

fbshipit-source-id: 1969f380f7ad13f54a7dbcd2283b0428c4480ab4
2018-06-04 18:23:03 -07:00
Durham Goode
6370628aeb treemanifest: move store logic to a separate function
Summary:
In a future diff we'll be adding a function that also needs to turn a
python store into a ManifestFetcher. So let's move that logic to a separate
function.

Reviewed By: quark-zju

Differential Revision: D8216728

fbshipit-source-id: f6d2e9577eccfcb015797a519264bd3b241ebae5
2018-06-04 18:23:03 -07:00
Durham Goode
18945e39b6 treemanifest: remove py_treemanifest from iterators
Summary:
Now that the subtree iterator doesn't contain a reference to the
treemanifest directly (it instead holds a ManifestPtr reference), we no longer
need to persist a reference to it at the python layer.

Reviewed By: quark-zju

Differential Revision: D8206301

fbshipit-source-id: 531d4399f0f54a47e0eb741f419e6242188a83ad
2018-06-04 18:23:03 -07:00
Durham Goode
aa1e1a5604 treemanifest: remove resultEntry from iterators
Summary:
Now that markPermanent no longer requires a ManifestEntry to update the
node, the resultEntry field on all the iterators is unused. Let's delete it.

Reviewed By: quark-zju

Differential Revision: D8156808

fbshipit-source-id: 21aeb050c91552c7e8973a33dd7e347457125a64
2018-06-04 18:23:03 -07:00
Durham Goode
4708242f44 treemanifest: remove logic that updates ManifestEntry.node
Summary:
Now that the node field is accessed via a getter that can lazily
compute it, let's get rid of the logic that proactively sets it. This will make
it easier to refactor the subtree iterator in a later diff.

Reviewed By: quark-zju

Differential Revision: D8156809

fbshipit-source-id: aabc3f68cc41baea0c166b9a2bd68bea6b1a6a03
2018-06-04 18:23:03 -07:00
Durham Goode
63834775d7 treemanifest: move ManifestEntry node behind an accessor
Summary:
Previously the ManifestEntry node field was public and consumers could
read it directly. This required that the node field be updated manually, which
added complications to other code paths. In a future diff we'll be simplifying
the SubTreeIterator code to not require setting node, and to do so we make node
calculated in this accessor.

Reviewed By: quark-zju

Differential Revision: D8156807

fbshipit-source-id: e1908b2149bc0ec8fb1279e37b8a66988ff125c6
2018-06-04 18:23:03 -07:00
Durham Goode
0bbc6b6666 treemanifest: switch native iterators to use ManifestPtr
Summary:
This switches the treemanifest native iterators to use the ManifestPtr
class instead of Manifest*. ManifestPtr does ref counting, and in a future diff
we'll use this ability to have the subtree iterator iterate over a portion of a
larger tree so we can serve just parts of trees to gettreepack requests.

Reviewed By: quark-zju

Differential Revision: D8156806

fbshipit-source-id: 4dbb60d008ac8d0c789c3f2db8f7e567c5869539
2018-06-04 18:23:03 -07:00
Saurabh Singh
7c9227818a refactor rust datastore to a consistent naming scene
Summary: This is just a refactor to address the naming scheme.

Reviewed By: quark-zju

Differential Revision: D8269217

fbshipit-source-id: 8c52d2c67837550e0b7dc1a45b3faf9a80319b61
2018-06-04 17:39:47 -07:00
Saurabh Singh
7067e4ca1f fix nit in implementation
Summary:
Based on review for D8214151 by quark-zju, addressing the nit here as
well.

Reviewed By: quark-zju

Differential Revision: D8267140

fbshipit-source-id: 12c3355852a49859c2b0a243fa8666105c914c73
2018-06-04 16:21:38 -07:00
Saurabh Singh
8ba5a79489 adding tests for bad data store
Summary:
Adding the tests for the case when the union store has only one data
store which always returns an `Err` as `Result`. This `Err` is not of the type
`KeyError` which the union store handles differently.

Reviewed By: quark-zju

Differential Revision: D8214156

fbshipit-source-id: bd077af343086c92f46ec6a6f1551d05dd9bda09
2018-06-04 16:21:38 -07:00
Saurabh Singh
242f2b904f add tests for empty data store
Summary:
Adding tests for the case when the union store only has a single data
store which is completely empty.

Reviewed By: quark-zju

Differential Revision: D8214151

fbshipit-source-id: 9d8f329548a1b7e105a5dc6219067a6e292fe97c
2018-06-04 16:21:37 -07:00
Saurabh Singh
34ca90a6f8 rename test methods to be more specific
Summary:
This commit just renames the methods to be more specific. This is
useful for later changes.

Reviewed By: quark-zju

Differential Revision: D8214153

fbshipit-source-id: e8db9148334f7cd539aca626e3798e256b9b022f
2018-06-04 16:21:37 -07:00
Mark Thomas
44fd2763b8 progress: add nullrenderer
Summary:
A new renderer that doesn't print anything.  This ensures the progress engine
continues to run (other ways of disabling progress completely disable progress
calculations), and can be used in conjunction with the progressfile extension
to give progress output there without anything on stdout.

Reviewed By: phillco, farnz

Differential Revision: D8258548

fbshipit-source-id: 78cb78d84c6d1dd22744d84d34bf0f43f7962ace
2018-06-04 14:07:30 -07:00
Phil Cohen
32421c6244 rebase: manually convert IMM metrics to strings
Reviewed By: singhsrb

Differential Revision: D8263053

fbshipit-source-id: 13c2e9449b9f43d12bb8e1a9582334740462ef4c
2018-06-04 13:51:59 -07:00
Mark Thomas
63dcc1c97e infinitepush: enable commitcloud extension in infinitepush tests
Summary:
We broke infinitepush with a commitcloud change.  This would have been caught
if we had the commitcloud extension enabled during the infinitepush tests.

Differential Revision: D8257849

fbshipit-source-id: 8a0da2e3268a9545502d432fcbce4b4c1eae5bcb
2018-06-04 11:19:09 -07:00
Liubov Dmitrieva
8983399a0e commitcloud bugfix
Reviewed By: markbt

Differential Revision: D8256953

fbshipit-source-id: 4009b770c628ebf80c15d3ccd23307f2589e7a12
2018-06-04 08:53:46 -07:00
Liubov Dmitrieva
9ebe334049 commitcloudsubscriber: enable new options for cloud sync
Reviewed By: markbt

Differential Revision: D8187376

fbshipit-source-id: cfc9feedb8763e36f2af8eec61d73f2e943b19d7
2018-06-04 06:47:22 -07:00
Liubov Dmitrieva
fd4980147d commitcloud: add / remove subscription
Summary: add / remove subscription to Commit Cloud Notifications (and notify the daemon to restart subscriptions)

Reviewed By: markbt

Differential Revision: D8186751

fbshipit-source-id: 7d726c8a29752793bf038c5bfc3a379c17712275
2018-06-04 05:38:22 -07:00
Durham Goode
ab72f94206 hgsql: add a sqlrefill command
Summary:
This adds a command that can be used to fill a database with data from
existing revlogs. This will be useful in the future for uploading new
repositories as well as fixing any database corruption.

In particular, this is important as a precaution to fixing the treemanifest
corruption.

Reviewed By: farnz

Differential Revision: D8108361

fbshipit-source-id: 20717c2057f2bed82c98323c4a3e9d0c6dbd7df5
2018-06-01 16:33:41 -07:00
Sergey Zhupanov
7d44cb5f01 Enabled additional compiler warnings in Eden.
Summary:
1. Enabled a number of additional C++ compiler warnings in Eden.
2. Fixed warnings-turned-errors that resulted from this change.

Reviewed By: simpkins

Differential Revision: D8132543

fbshipit-source-id: 2290ffaaab55024d582e29201a1bcaa1152e6b3e
2018-06-01 11:10:54 -07:00
Mark Thomas
e6e191d17f debugcheckcasecollision: fix type of -r default value
Summary:
The default value of `rev` for `debugcheckcasecollisions` should be `""`, not
`[]` (the latter causes an exception).

Differential Revision: D8236465

fbshipit-source-id: 841e0218664c46e70abd5b774463d4b4e40fdfb8
2018-06-01 09:32:57 -07:00
Jeremy Fitzhardinge
98be816aba rust/tp2: update rust-crates-io
Summary:
Big change here is update to rand 0.5. This is a significant API
change. quickcheck still uses rand 0.4, so for quickcheck users I changed it so
that quickcheck re-exports the rand it uses. This means that quickcheck users
are unchanged aside from using quickcheck::rand, whereas direct rand users have
been updated to use the new API.

Reviewed By: farnz

Differential Revision: D8234503

fbshipit-source-id: f9e620851b8dfcc33f22a0af26122adcd5fbde39
2018-06-01 09:32:56 -07:00
John Kearney
a2ea2bde8f Fix windows hg merge (for editors other than vim and notepad++)
Summary: Fix handling of quoting inside editmergeps.ps1

Reviewed By: ikostia

Differential Revision: D8224020

fbshipit-source-id: eaeb0fd6dcd5b2aef8169edd8eef4618dfb0de52
2018-06-01 08:40:11 -07:00
Phil Cohen
6785da2366 fix flaky test-fb-hgext-remotefilelog-sparse.t
Summary: Similar to D8232838. This test can be so slow on Legocastle it fails on the leading 0.

Reviewed By: quark-zju

Differential Revision: D8233716

fbshipit-source-id: 7b617caacbea4a7ae244cf4cd7568afff0628a5e
2018-05-31 21:50:09 -07:00
Adam Simpkins
bd9208583d tests: fix test-check-clang-format.t to use unique temporary names
Summary:
Update test-check-clang-format.t to put its output files in "$TESTTMP" rather
than using fixed paths in the source repository.

This should fix stress test runs of this test, which run multiple copies of
the test in parallel.  Previously running several copies of this test in
parallel would fail as they all try to use the same output path names.

Reviewed By: quark-zju

Differential Revision: D8229266

fbshipit-source-id: a3728aaae7fa8ff393835e7edf09abf2e3939b69
2018-05-31 21:04:43 -07:00
Jun Wu
190087e4a2 hint: pick the user config file that exists
Summary:
When running `hg hint --ack ...`, pick the file that exists to write,
instead of using the first default one.

Reviewed By: singhsrb

Differential Revision: D8230173

fbshipit-source-id: 90375653ffa16cfbf77eed4f967b5250b649a532
2018-05-31 20:16:07 -07:00
Saurabh Singh
f1b5dd5d1c fix flakiness in test-http-bundle1.t
Summary:
This test fails if the transfer rate is in bytes/sec instead of KB/sec.
It's best to ignore KB or bytes to make the test less flaky.

Reviewed By: phillco

Differential Revision: D8232838

fbshipit-source-id: 5340fa0cfef4c7c72839dc71e7bf21c138be417e
2018-05-31 20:16:07 -07:00
Liubov Dmitrieva
f6de146d60 commitcloud: add hint for updateonmove option
Reviewed By: quark-zju

Differential Revision: D8204928

fbshipit-source-id: 2bdf1fed3416be2f1a89355020208022327b6524
2018-05-31 18:56:28 -07:00
Adam Simpkins
4ff77abfa6 speed up hg revert on large directories
Summary:
When called with any matcher parameters, `hg revert` would first walk both the
working directory and the commit in question to find all files that match the
pattern.  Only after it found all matching files did it check to see which
ones actually need updating.

This made revert performance `O(matched files)` rather than
`O(files needing revert)`.  This makes operations like
`hg revert large_directory` quite expensive, especially when only a handful of
files need to be updated.

This should help address the performance of `hg revert .`, which previously
was much slower than `hg revert --all`.  Previously it enumerated every file
in the repository twice (once in the working directory and once in the current
commit) before checking which ones needed reverting.  This should also help
fix the performance of `hg revert <directory>` in Eden as well.

Reviewed By: quark-zju

Differential Revision: D8217972

fbshipit-source-id: c18e72c8a5ed0b4c3a5ed69e79f6a88d3ea04a9f
2018-05-31 17:30:32 -07:00
Adam Simpkins
528693e7be context: fix workingctx.status() to invoke match.bad correctly
Summary:
Calling workingctx.status() ignored the input matcher's bad() function and
always replaced it with its own function intended to filter out bad() calls
for directories that do exist on disk.  However for paths that were
legitimately bad it directly printed a message using ui.warn() rather than
calling the original input matcher's bad() function.

This updates workingctx.status() to invoke the input bad() callback so that it
behaves like the `status()` method on other context object types.

Reviewed By: quark-zju

Differential Revision: D8217973

fbshipit-source-id: 9dd458b96bc43e326791873e599dc38a93898c58
2018-05-31 17:30:31 -07:00
Durham Goode
4d9d0a91dc treemanifest: avoid ondemand conversions on the tree server
Summary:
We were encountering issues where readonly connections where failing
because the serve was returning a Permission denied error while fetching trees.
It turns out the ondemand tree generation was attempting to open a transaction
and create trees on the server.  Since the server should have all the trees
anyway, we shouldn't do this ondemand conversion.

We still need on demand conversions for infinitepush bundles, so we'll leave
that in.

Reviewed By: singhsrb

Differential Revision: D8226459

fbshipit-source-id: 2d05faefbe3ae92ccb81072e012673f29c9cf0d9
2018-05-31 14:54:57 -07:00
Durham Goode
1af4bd9dbe format: auto format treemanifest
Summary: Some formatting issues got in.

Reviewed By: phillco

Differential Revision: D8226461

fbshipit-source-id: 04b3fc7245e13e93f165024ad1601349a05e3d93
2018-05-31 14:54:57 -07:00
Phil Cohen
eb1f92e6ec repogenerator: allow starting rev to be configured
Summary: Sometimes it's useful to be able to resume generation from a commit that isn't the tip (e.g., after recloning or pulling).

Reviewed By: singhsrb

Differential Revision: D8217644

fbshipit-source-id: fd622ed20e1e3782f3bc3e2df58a2d33425ea496
2018-05-31 12:33:26 -07:00
Wez Furlong
a829ad5c5c hg: maybe make hgsubversion respect the sharedpath
Summary:
I'd like to make it so that eden clones don't need to
rebuild the svn metadata on the first operation after a clone,
and don't need special post-clone logic like `hg-new-workdir` has.

Reviewed By: quark-zju

Differential Revision: D8219138

fbshipit-source-id: da1e9c9a1389893ecd49962ad3143445e85fb247
2018-05-31 09:10:27 -07:00
Adam Simpkins
641bac8427 improve cdatapack fanout table calculation
Summary:
Refactor the cdatapack logic that computes the fanout table.  This more
accurately computes the correct ranges to bisect for each fanout table entry.

This fixes an off-by-one error setting end_index in most buckets that caused
it to search a slightly larger bisection range than necessary.

This also fixes the code to accurately compute which buckets do not have any
nodes, and sets a (start, end) range of (1, 0) for these buckets, causing
find() to avoid having to search anything in these cases.

Reviewed By: quark-zju

Differential Revision: D8131019

fbshipit-source-id: 70d6d0f2e1d900a2df27b64f3a38f114d301be0d
2018-05-30 18:54:19 -07:00
Durham Goode
94115ed589 truncate: handle bug in truncate
Summary:
btrfs has a bug involving truncate not returning the correct error
code. We already handle it in other cases, so let's extend it to these uses of
truncate. Hopefully the need for this disappears soon.

Reviewed By: quark-zju

Differential Revision: D8152297

fbshipit-source-id: f55602ff5e0ec36346c547bfd4b6d0f6e4127500
2018-05-30 18:20:57 -07:00
Durham Goode
1aa9db8542 treemanifest: include p2 when creating trees
Summary:
Previously we weren't including p2 when creating trees, this resulted
in incorrect conversions.

Reviewed By: phillco

Differential Revision: D8152253

fbshipit-source-id: c4f8c79b40532c5162b15032962fbc6a78d44b5f
2018-05-30 18:20:57 -07:00
Durham Goode
06e89ef39e treemanifest: check memcmp < 0 instead of memcmp == -1
Summary:
The memcmp contract is that the result is >0, 0, or <0. If <0 it's not
guaranteed to be -1, so let's broaden the check a bit.

Reviewed By: phillco

Differential Revision: D8190843

fbshipit-source-id: 78b864a639851a041c1a393f21a979d13eaa2e9c
2018-05-30 18:20:57 -07:00
Durham Goode
2ad3a4421e treemanifest: switch tree diffs to using the fast path
Summary:
This switch converting flat manifests to tree manifests to use a fast
path where we avoid decompressing the whole manifest whenever possible. This
will speed up tree conversions tremendously.

Reviewed By: phillco

Differential Revision: D8130357

fbshipit-source-id: aa91f9445216d2c5cb60d51c21ae0f6636359d9c
2018-05-30 18:20:57 -07:00
Mark Thomas
e28d4a84e0 fix tests after reformat
Summary: The recent reformat caused a few test failures, this should fix them.

Differential Revision: D8207359

fbshipit-source-id: ced37aff3038715e5876fe27b58a60ebe45e4097
2018-05-30 13:47:50 -07:00
Adam Simpkins
2ea8028d14 fix a bug in cdatapack fanout table calculation
Summary:
If all of the nodes in a datapack file start with the same byte value,
the cdatapack code computed the fanout table incorrectly.  If this byte value
was anything other than 0x00 it would only be able to find the last node in
the pack, and would not be able to find any other nodes.

This fixes the code to compute the fanout table correctly in this case.

Reviewed By: quark-zju

Differential Revision: D8131020

fbshipit-source-id: 84e49befc5776cff96831f6120194466d9c80b35
2018-05-30 13:47:50 -07:00
Liubov Dmitrieva
4727ecd46d commitcloudsubscriber: improve code
Summary: minor code improvements

Differential Revision: D8206294

fbshipit-source-id: 7ea46db7b7af200665b84d00f8912fa385ebc091
2018-05-30 13:47:50 -07:00
Liubov Dmitrieva
d865e06a0d commitcloudsubscriber: add log throttling rates and add tcp socket listener to receive simple commands
Summary:
Added logic to control logging rate: empty messages that comes to confirm the subscription is alive, also on error logging rate when we are offline, also when we are running in standby with no active subscriptions

Also, I made a simple cross platform API, so that hg can trigger restart subscriptions in 2 lines of code. It is simple request - response API on tcp socket and json.

If a human run `hg cloud join`, hg will add subscriber file to the directory scm daemon reads subscribers from and will send the restart command, same for any `hg cloud leave` run

Another advantage is that the client (hg) can very easy check if the scm daemon is alive or not. (In 2 lines of code, cross platform, without any pid logic or other platform specific ifs)

Another advantage is that we can use it to receive some stats from the scm daemon.

I decided do not go with any watching directory logic, because changes are really rare events, and it will be better if a client (hg)  will just notify the service to restart subscriptions when needed.

Also, I verified that hg and SCM Daemon use the same config options and logic related to detected home directory on different platforms and reading the token.

Reviewed By: markbt

Differential Revision: D8162237

fbshipit-source-id: 3cb48b90f5e065ce4dc7fdc7215c3ce6ad57fb9a
2018-05-30 08:15:17 -07:00
Lukasz Langa
dfda82e492 Upgrade to 18.5b1
Summary: Mostly empty lines removed and added.  A few bugfixes on excessive line splitting.

Reviewed By: quark-zju

Differential Revision: D8199128

fbshipit-source-id: 90c1616061bfd7cfbba0b75f03f89683340374d5
2018-05-30 02:23:58 -07:00
Durham Goode
bb3dd10939 treemanifest: change _getflatdiff to output added/removed
Summary:
This makes the signature match _getfastflatdiff, so we can unify them
in the next diff.

Reviewed By: phillco

Differential Revision: D8130358

fbshipit-source-id: 4bc51b79d0432b3347dfd7996c9f586f0c65999f
2018-05-29 15:36:30 -07:00
Durham Goode
67e797a85d treemanifest: refactor new tree creation into separate parts
Summary:
This refactors the new tree creation logic from a single function to
two functions, one that computes the differences in (added, removed) format, and
another that accepts adds/removes and applies them to a copy of the parent tree.

This will help in a future diff where we want to unify code paths to use the
fast diff path.

Reviewed By: phillco

Differential Revision: D8122900

fbshipit-source-id: 82699dc4bbf69dd565f8564e71f8a0726659313e
2018-05-29 15:36:29 -07:00
Durham Goode
8cfdac7b77 treemanifest: move fast newtree computation to a function
Summary:
We already had logic that could compute a new tree from a manifest node
quickly (using the manifest revdiff function to avoid reading entire manifests).
Let's refactor that to a function so in a future diff we can unify it with
_getflatdiff so all paths can take advantage of the performance.

Reviewed By: phillco

Differential Revision: D8122469

fbshipit-source-id: 5cb315327356cece9b2fed3e2c57088179ef838e
2018-05-29 15:36:29 -07:00