Commit Graph

59842 Commits

Author SHA1 Message Date
Jun Wu
d047f07b70 commits: add a trait to describe storage backend and use-cases
Summary: This will be used to describe what the commit graph backend is.

Reviewed By: sfilipco

Differential Revision: D22970577

fbshipit-source-id: 753efdbdd4466730ece758d9f4789fbd21e2801b
2020-08-21 13:00:45 -07:00
Jun Wu
b77355ca0c commits: add double write commits backend
Summary:
This allows us to try segmented changelog while maintaining revlog
compatibility.

Reviewed By: sfilipco

Differential Revision: D22970583

fbshipit-source-id: 7c43cdadd76300e76e89f38aac5ed3ecc0cff728
2020-08-21 13:00:45 -07:00
Jun Wu
c785e333af remotefilelog: do not resolve linkrev for changelog.add
Summary: The value of linkrev is invalid with segmented changelog. Do not resolve it.

Reviewed By: sfilipco

Differential Revision: D23036079

fbshipit-source-id: 8f8b097458bc694327db6ba4e2dc4107bdf44157
2020-08-21 13:00:45 -07:00
Durham Goode
a2940dbbde py3: fix hex encoding for phrevset in git backed repos
Summary:
Phabricator reports some repos as being backed by git, and the phrevset
code path in that case uses the python 2 hex encoding. Let's just use the hex
function directly.

Reviewed By: quark-zju

Differential Revision: D23207771

fbshipit-source-id: 88de3153e52a3db456c17ab4ca4b9c9dd6049855
2020-08-19 10:53:12 -07:00
Viet Hung Nguyen
dcb565409d mononoke: Sync repo.thrift changes to fbcode
Summary:
Related commit: D23213192

Currently repo configs don't store their phabricator callsigns and because of this,
it's hard to generate them with a given repo name (e.g. fbsource) in Mononoke.
This diff synced the changes between configerator (check the other commit) and fbsource.

Reviewed By: farnz

Differential Revision: D23214677

fbshipit-source-id: cf102d69958c7dc32f8b72cecbfad017d0a31329
2020-08-19 10:48:41 -07:00
Stanislau Hlebik
666182b451 mononoke: add one more function to create DifferenceOfUnionsOfAncestorsNodeStream
Summary:
In the next diff I'd like to compute generation number first, and then call
DifferenceOfUnionsOfAncestorsNodeStream. To avoid refetching these numbers
again let's create a function that accepts a vector of (ChangesetId,
Generation) pairs.

While here I also made the order more consistent: now we have "hashes"
parameters always in front of "excludes"

Differential Revision: D23212883

fbshipit-source-id: 11e0a1494126f84b36e3e33e65071449db5840d2
2020-08-19 07:30:11 -07:00
Lukasz Piatkowski
411a03ee58 mononoke/integration tests: fix tests after moving dummyssh (#43)
Summary: Pull Request resolved: https://github.com/facebookexperimental/eden/pull/43

Reviewed By: StanislavGlebik

Differential Revision: D23211512

Pulled By: lukaspiatkowski

fbshipit-source-id: 5c831c0d3dfd595647138f98968b91c1660c0856
2020-08-19 04:09:54 -07:00
Stanislau Hlebik
30150b244e mononoke: log correct file size for undesired file fetches
Summary:
Previously for undesired fetches of lfs files we were logging 0. Let's log the
real path instead

Reviewed By: ikostia

Differential Revision: D23209754

fbshipit-source-id: 7a893b257a89332a5169ab2072ecf48ae94b91e0
2020-08-19 02:41:19 -07:00
Mark Thomas
f64f57eed1 debuginstall: don't fail if no templates are installed
Reviewed By: quark-zju, ikostia

Differential Revision: D23154918

fbshipit-source-id: 18193103e0c765ec7064bb67b39548d32b289366
2020-08-19 00:32:19 -07:00
Mark Thomas
44b7724014 run-tests: add internal:none editor
Summary: Rather than using a Python program, simulate having no editor during tests using `internal:none`.

Reviewed By: quark-zju, ikostia

Differential Revision: D23152446

fbshipit-source-id: 5560f58885ee5959c62f0ac8bcf0483b9c3072f6
2020-08-19 00:32:19 -07:00
Mark Thomas
a25521a48b tests: remove uses of $PYTHON -c
Reviewed By: quark-zju, ikostia

Differential Revision: D23151936

fbshipit-source-id: f5dc1bf1e2007fe82c8ea67fa94c7b5aa3fd9a0c
2020-08-19 00:32:19 -07:00
Durham Goode
d31488e3fa py3: fix commandserver args to be string
Summary:
The commandserver was setting args as bytes when they should be
strings. This was breaking arg parsing for users using hglib, which communicates
with hg via the commandserver.

Reviewed By: sfilipco, singhsrb

Differential Revision: D23202623

fbshipit-source-id: f71e4145211069b2f7ed6935fe86585061cbf8b3
2020-08-18 21:36:47 -07:00
Jun Wu
324e79fac4 setdiscovery: enforce new discovery logic for rust changelog
Summary:
The old discovery logic is incompatible with the rust changelog because
the changelog is not based on revlog.

Reviewed By: sfilipco

Differential Revision: D23036065

fbshipit-source-id: 633fdf8726d40cb14e63c3df2f5573d35cb1640a
2020-08-18 20:43:29 -07:00
Jun Wu
0b0cd2da97 setdiscovery: use new dag APIs if possible
Summary:
They are faster if the new dag backend is used. For example, `headsancestors`
will be backed by a fast native path, where the `head(ancestors(x))` revset
would be unusably slow if the new dag backend is used.

Reviewed By: sfilipco

Differential Revision: D23036069

fbshipit-source-id: 75df1e2240520a6e560bc5d8414ba2f4fb7a4674
2020-08-18 20:43:29 -07:00
svcscm
96cb6dc0a5 Updating submodules
Summary:
GitHub commits:

2f42bd0f67
adea16795e
9543987f4a
1a5201135d
c0377a45a0
74db414168
ab1d231926

Reviewed By: jurajh-fb

fbshipit-source-id: acde159c799ae829145b6b0f4fcf967d3025f626
2020-08-18 15:12:57 -07:00
Durham Goode
6632d69936 py3: fix hg doctor in shared repos
Summary:
It read the sharedpath manually but needed to read it as utf8, not
bytes.

Reviewed By: quark-zju

Differential Revision: D23194858

fbshipit-source-id: aa73c2cc782070ba6c4d2c441d6f5338842b137f
2020-08-18 14:57:51 -07:00
Durham Goode
59f867a442 ssh: prevent double cleanup of sshpeer
Summary:
sshpeers were being cleaned up twice. Once during the explicit close()
and once during the __del__ destruction. This caused it to double count the
reported _totalbytesreceived.

Reviewed By: quark-zju

Differential Revision: D23178435

fbshipit-source-id: da76158be561511f2f2a5e255fd03aa4a6b78da0
2020-08-18 14:49:58 -07:00
svcscm
c779d87aa9 Updating submodules
Summary:
GitHub commits:

8435420c95
d080d6af2c
d2794b2710
16a708482b
1c4d87714f
8e45ce8389
70c1919557
1d5d0e1b4f
c8224606a8
31a5160a73
50424a6bf9

Reviewed By: jurajh-fb

fbshipit-source-id: 23f88ca16c867b5c203348c90637b06ea091d6af
2020-08-18 14:45:42 -07:00
Durham Goode
d7b036c29a Enable fb features for cargo test diff runs
Summary:
We missed a Windows http client breakage because our LFS server integration
wasn't run on Windows. Let's enable the fb feature for all our cargo test runs.

Reviewed By: singhsrb

Differential Revision: D23140315

fbshipit-source-id: 46cc533c1e543ffc32d472b49a8f6daeee3b5009
2020-08-18 14:01:01 -07:00
Meyer Jacobs
656e3c90d6 edenapi: Introduce serde annotations for wire protocol compatibility and compact wire representation
Summary:
Aux data wire protocol part 1: field annotations & basic compatibility model.

Annotates fields in `file`, `tree`, and `complete_tree` wire structs with `#[serde(rename = "N", default, skip_serializing_if = "is_default")]`. I've avoided using `#[serde(default)]` on the container structs themselves because this can cause some confusion / incorrect behavior if not used carefully. Consider a wire struct `FooRequest` with a field of type `Option<Bar>`. `Option<Bar>` defaults to `None`. If `FooRequest`'s `Default` implenentation sets the field's default to `Some(bar)`, a `FooRequest` explicitly constructed with `None` for the field will be serialized with the field omitted (because it passes `is_default`) and will be deserialized on the server as `Some(bar)`, causing incorrect behavior. To address this, we'd need to change the `is_default` function used with `skip_serializing_if` to check against the field's default value as set by the container, which isn't trivially possible without some sort of reflection (please correct me if you know a good way to achieve this). This is unfortunate, as it'd be very desirable for the container to be able to set defaults different from the individual field type defaults, for cases where one boolean, for instance, should default to true. As-is, we'd need to address this with wrapper types instead, where we can fully control the `Default` implenentation.

We can, of course, address this by providing an alternate `skip_serializing_if` function to fields with default that doesn't match that set by the container. This will need to be done carefully, though, to avoid the issue I described above.

Currently the JSON module manually serializes and deserializes all the top-level request objects, so the rename annotation doesn't impact it. We can add `#[serde(alias = "rustfieldname")]` if we'd like the server and client to be able to accept manually-crafted requests and responses with explicit field names. This could also be useful to replace the manual parsing in the JSON module, but can't replace the manual serialization in a clean way. We'd need to introduce a second copy of the wire types, without the serde `rename` attribute, to allow serializing with the actual rust field name.

I've only modified the `tree`, `file`, and `complete_tree` modules. I intend to eventually update the rest of the edenapi protocol later on, when the implementation of `file` and `tree` are complete / stable. This will give us a chance to fix any mistakes before copying the design to more places.

Note: I do not intend to keep to proper wire protocol compatibility at this stage in the implementation. Expect field numbers to be re-used by non-compatible changes.

Reviewed By: kulshrax

Differential Revision: D23172756

fbshipit-source-id: 39976ed4bede892bd6981f9c3f23557a91f9028b
2020-08-18 13:44:35 -07:00
Jon Janzen
3e9ba05f80 Add github action to auto-release TagIt tags
Summary:
This came from a [request from Fred Emmott](https://fb.workplace.com/groups/osssupport/permalink/4116491528399431/), but also is something we should do due to the GitHub UI redesign:

Nowadays on GitHub, tags are not shown in the sidebar (just the count) and instead GitHub Releases are prioritized and the most recent *release* is shown.

Secondly, the Source Code zip archives for tags/releases are created on-the-fly on GitHub. This can cause problems for downstream projects that need consistent content hashes of the archive for security/reproducability reasons. This action also creates its own archives (zip and tar.gz) and attaches them to the release.

Reviewed By: fredemmott

Differential Revision: D23167073

fbshipit-source-id: 463e9d93a2c4af260ac3c4dbc1ceab7894add714
2020-08-18 13:31:10 -07:00
svcscm
5e8a950da0 Updating submodules
Summary:
GitHub commits:

ee03860b54
6398dab42e
77f6fc4926
e381336eaf
f5cc16728e
f234fb355a
26432374d9
cbf9d75393
b6bae911e5
aebf28b412
bb5e92bfc0
4c1510987a

Reviewed By: jurajh-fb

fbshipit-source-id: cc5891685a5be9cd76e48c3c4eaa8f6272f87f2c
2020-08-18 12:55:36 -07:00
Jun Wu
f8d9e51bc9 github: rename "EdenSCM HG" to "EdenSCM"
Summary: Make it consistent with the top-level README which does not use the "HG" name.

Reviewed By: kulshrax

Differential Revision: D23176134

fbshipit-source-id: 2d809cfa89234d00c714f9db381a138e9f03fc64
2020-08-18 12:39:08 -07:00
svcscm
240f114c30 Updating submodules
Summary:
GitHub commits:

26762fea56
b982330ba3
dff1cabebc
cd9669afa3
1273272066
e6e2f3699c
446d55fb3b
780add8e18
26b9aaed17
0b8b6ae581
f234559728
fb91abaac9
c52c1fa8f1
1418add150

Reviewed By: jurajh-fb

fbshipit-source-id: d70f15db82d44f99f4c1e2f4c6a2402d3a09e1a4
2020-08-18 10:58:04 -07:00
Liubov Dmitrieva
a9b5e8668b Add create option and validate that workspace exists
Summary:
hg `cloud join -w myworkspace --switch/--merge` should require --create option for all not default
workspaces if the workspace doesn't already exist.

I also enabled reusing of http connection, so it makes `hg cloud switch` much faster.
It is less than 2 sec now.

Reviewed By: markbt

Differential Revision: D23188491

fbshipit-source-id: 9a2ba2df4ba91e9dba9484cd49fc886f70b09880
2020-08-18 09:46:22 -07:00
Meyer Jacobs
a01e4e3503 filemerge: Fix regex to use bytestring pattern for bytestring data.
Summary: In py3, a string pattern cannot be used to match against a bytestring text. Convert the pattern to a bytestring, which should be compatible with python2 without additional changes.

Reviewed By: DurhamG

Differential Revision: D23135472

fbshipit-source-id: fcc0b111c646011f5b556f032ef85ef326cb9511
2020-08-18 09:14:07 -07:00
Viet Hung Nguyen
e1b088d315 mononoke/repo_import: moved tests
Summary: Minor refactor: moved the tests from main to their own file, because main was getting too large and found it hard to navigate through the code.

Reviewed By: StanislavGlebik

Differential Revision: D23188766

fbshipit-source-id: a2b2e32c77587f95c07a0bb02a4957e3671dd2c6
2020-08-18 09:09:14 -07:00
Johan Schuijt-Li
4e0660a94c asyncify connection accepting
Summary:
This largely moves connection accepting from old style bytes, futures and tokio
to updated versions, while keeping some parts at old bytes/futures in order to
remain compatible with the rest of the Mononoke codebase.

Division lies on `Stdio` which maintains old channels, stream and futures,
while the socket handling, connection acception and wire encoding is updated.

With the updated futures, we now wait for the forwarding stream to have
succeeded before considering a connection fully handled.

Other notable changes:
 - futures_ext now a mini codec Decoder instead of relying on NetstringDecoder,
   which has been updated to use bytes 0.5
 - hgcli has been modified to use updated NetstringDecoder
 - netstring now requires the updated bytes 0.5 crate
 - the part in connection_acceptor was handling repo/security logic is now part of repo_handler (as it should have been), connection_acceptor now only handles networking and framing
 - tests now verify that the shutdown handler is triggered

Reviewed By: krallin

Differential Revision: D22526867

fbshipit-source-id: 34e43af4a0c8b84de0000f2093d7fffd3fb0e20d
2020-08-18 09:09:14 -07:00
Koray Polat
ad6393d8d4 Add option to run tests without Testpilot even when it's available
Summary:
- Added a commandline flag to ignore testpilot even when it's available
- Fixed an oversight that caused getdeps to return 0 even if ctest tests have failed.

Reviewed By: wez

Differential Revision: D23161362

fbshipit-source-id: 1ed97c481315e2b39f1128680386390930194970
2020-08-18 09:06:53 -07:00
svcscm
ea519fe4d9 Updating submodules
Summary:
GitHub commits:

696d36e4c8
d6e4753c2c

Reviewed By: jurajh-fb

fbshipit-source-id: 7468f781c81566ca01037e4d9c2c9a04f4de82de
2020-08-18 09:06:53 -07:00
svcscm
ab3a42d278 Updating submodules
Summary:
GitHub commits:

475f4d573d
93b4b9ee46
8766680a2f
5e4f665cd6
e72195fd60
32837139c6
92593d511a
e363323bdd
f1dbf24b94
5a2fa04d2a
9daffaea48
ab6ea35582
25a061b5c9
8fe2b7d60c
ba0384954a

Reviewed By: jurajh-fb

fbshipit-source-id: 4aa31bbb4718c847131ad9521ad71be1a0b16cf7
2020-08-17 18:14:32 -07:00
Chad Austin
c12f60afb5 only use symbolizer if libunwind is found
Summary: folly/experimental/symbolizer requires libunwind. Do not enable it unless libunwind is available.

Reviewed By: yfeldblum, luciang

Differential Revision: D22964401

fbshipit-source-id: d71991346927fbf18167637770b2143c03b16476
2020-08-17 17:08:55 -07:00
Xavier Deguillard
88c3bf4826 revisionstore: remove translate_lfs_missing
Summary:
As noted in the documentation for it, this can be removed once get and prefetch
return a continuation. This is now done, and thus we can remove it entirely.

Mis-use of it caused data to be fetched twice: once by memcache, and the second
one by getpackv2.

Reviewed By: singhsrb

Differential Revision: D23123344

fbshipit-source-id: 9ac0594faaba94ead04a8bb9035e14809a706641
2020-08-17 17:05:58 -07:00
Durham Goode
fe6cb9dc13 configs: fix handling shared path with trailing new lines
Summary: The python code stripped new lines but the Rust code did not.

Reviewed By: singhsrb

Differential Revision: D23167515

fbshipit-source-id: add33ec6e4cfd9169e6fef8208490e0aeede38bd
2020-08-17 15:53:08 -07:00
Durham Goode
36a49b9b2b rotatelog: fix tests on windows
Summary:
The tests weren't windows compatible. Let's get rid of the path
separators.

Reviewed By: quark-zju, singhsrb

Differential Revision: D23170573

fbshipit-source-id: f934691cd2891205442885a12debe3a28d275fc5
2020-08-17 14:08:26 -07:00
Mark Thomas
526aac3aa2 scribe_commit_queue: include received_timestamp
Summary:
Subscribers to the commit tailer categories would like to know when Mononoke
received the commit.

Reviewed By: StanislavGlebik

Differential Revision: D23162447

fbshipit-source-id: 747214f1964a643f59c491aa08cdbd5c8fe331c8
2020-08-17 13:13:12 -07:00
svcscm
e29b75beae Updating submodules
Summary:
GitHub commits:

562bbaa2c7
6d492e885d
227f041e35
49b1f90bdd
1babfa857a
919b390d81
0ebfa43666
37050483ca
c68a800622
d6729b40bd
7b85e28fa8
3eaffcca42

Reviewed By: jurajh-fb

fbshipit-source-id: 657232b45379446deaa3a8ec8605aaecce9ad163
2020-08-17 11:57:19 -07:00
Jonathan Kim
5cb2331899 Update folly's source of truth to xplat
Reviewed By: mzlee

Differential Revision: D23066827

fbshipit-source-id: 6362acb7a6680a24871859d7874a9caec0a1538e
2020-08-17 11:01:52 -07:00
Liubov Dmitrieva
e6509babef enable automated migration from old backups to cloud sync
Reviewed By: markbt

Differential Revision: D22802064

fbshipit-source-id: c56e65e1a93e8cc94296cf69227bc51eb52f59c3
2020-08-17 10:58:38 -07:00
Mark Thomas
23dee0c931 mononoke_api: add freshness parameter to resolve_bookmark
Summary:
Allow callers of `resolve_bookmark` to specify whether they'd like the most recent value of
the bookmark, rather than one which may be stale.

Use this in the repo_move_bookmark test to avoid flakiness caused by the test code racing against
the warm bookmark cache.

Reviewed By: StanislavGlebik

Differential Revision: D23151427

fbshipit-source-id: 4b8358be1cf103479ccc23a41b2505776543ee49
2020-08-17 09:09:07 -07:00
Mark Thomas
053abf7919 hook_manager_factory: extract construction of the hook manager
Summary:
Extract construction of the hook manager to its own crate, so that we can re-use it.

Eventually the hook manager will become a repo attribute and will be constructed by
the repo attribute factory, but for now it needs its own factory method.

Differential Revision: D23129407

fbshipit-source-id: 302fde4d1ae38c6f61032a32c880018ebf84dee2
2020-08-17 09:09:07 -07:00
Mark Thomas
563137e6f7 hooks: create HookRejection struct
Summary:
Convert hook rejections from a tuple to a named struct.  This will be used in
the bookmarks_movement public interface.

Reviewed By: krallin

Differential Revision: D23077550

fbshipit-source-id: a35476817660c38b8df879ba603b927a7e39be21
2020-08-17 09:09:07 -07:00
Viet Hung Nguyen
d3f3cffe13 mononoke/repo_import: check if repo pushredirects
Summary: Some repos are push-redirected repos: pushes go to another repos, but then synced into this repo. Because of this, when we import a repo into a smaller repo that push-redirects to a large repo, we need to make sure we don't break the large repo with the imported code, since merges, pushes, imports etc. are redirected to the large repo. For now, in order to avoid breaking the large repo, we added a simple check that returns error, if the small repo push-redirects to the large one.

Reviewed By: ikostia

Differential Revision: D23158826

fbshipit-source-id: f722790441d641f67293e78c5d1ea5d1102bbb9b
2020-08-17 06:13:21 -07:00
svcscm
7b916a0c52 Updating submodules
Summary:
GitHub commits:

e4be93cbb8

Reviewed By: yns88

fbshipit-source-id: 6cd4078dcd2652904f5a8fa0a84b8b83c4564f21
2020-08-17 03:13:33 -07:00
Mark Thomas
2117806dc7 tests: convert dummyssh and get_free_socket to python3
Summary: Convert dummyssh and get_free_socket to full python3 binaries.

Reviewed By: johansglock

Differential Revision: D23105490

fbshipit-source-id: 6c39c32ba0728cde108b42245acece1d7828ac7c
2020-08-17 02:42:14 -07:00
svcscm
166a45e975 Updating submodules
Summary:
GitHub commits:

c8aa853cb5
2436435602

Reviewed By: yns88

fbshipit-source-id: 9438ece6b6a91a723d7fb9b16488b4984865f783
2020-08-17 02:42:14 -07:00
svcscm
32f5a2a7fd Updating submodules
Summary:
GitHub commits:

9fe9e1ed32
72530db795
97ea5d6bb1
516b76e65e
37c450a75b
3d3c2f89f5
5545fd377c
f50740edd8

Reviewed By: yns88

fbshipit-source-id: 651f44eded2da6d7b96d886a018a60c81578aed2
2020-08-16 20:43:44 -07:00
Durham Goode
33a634167e dynamicconfig: support a disallowlist config
Summary:
This new disallowlist will let us specify config section.key's which
should not be accepted from old rc files. This will let us incrementally disable
loading of those configs from the old files, which will then let us delete them
from the old rc's and eventually delete the old rc's entirely.

This diff also removes hgrc.local and hgrc.od from the list of configs we
verify, since those are not on the list of configs that need to be removed in
this initiative.

Reviewed By: quark-zju

Differential Revision: D23065595

fbshipit-source-id: 5cd742d099efd651174cab5e87bb7cdc4bae8054
2020-08-16 16:56:00 -07:00
Durham Goode
0cf7ebeffe configs: make backingstore load hg configs through the approved path
Summary:
Previously the backing store was loading configs manually. Now that
system, dynamic, user, and repo config loading are unified, let's go through
that approved path.

Reviewed By: kulshrax

Differential Revision: D22736338

fbshipit-source-id: 232023e660107a096691e9d99bf89c04c218dfbd
2020-08-16 16:56:00 -07:00
Durham Goode
6da00020eb configs: move all dynamic and repo config loading out of Python
Summary:
The last few diffs prepared Rust for loading dynamic and repo configs.
This diff finally changes Python so that we're no longer doing any of this work
in Python.

Reviewed By: quark-zju

Differential Revision: D22712625

fbshipit-source-id: 0b71e81d79d10ea3ce7a1b31f315ada5728af9a9
2020-08-16 16:56:00 -07:00