Summary:
Previously, we assumed that all content hashes came from Mercurial;
this is not going to remain true, as we will want to be able to upload manifests
that have been synthesised from Bonsai Changesets. Turn the previous boolean
into a tri-state, and fix up all callers to get the behaviour they expect.
Reviewed By: StanislavGlebik
Differential Revision: D8014911
fbshipit-source-id: 9156b9fab4542ceb269626ad005e1b28392b5329
Summary: Parsing and reading revlogs is cpu intensive, thus let's use cpupool for it
Reviewed By: StanislavGlebik
Differential Revision: D7926174
fbshipit-source-id: 7f023088941e1ad118a683da972f87607e0bfec4
Summary: printing every CS is too verbose, but we still want to see progress in non-debug mode
Reviewed By: kulshrax
Differential Revision: D7925747
fbshipit-source-id: c3ed92ef8c8fbf7714779a2bf011d31c94aefa37
Summary:
Rust 1.26 adds many new language features. In particular `impl Trait` is now
stable, so we no longer need `conservative_impl_trait`.
There also seems to have been changed in the (unstable) TryFrom with respect to
usize, and the behaviour of the never type `!`.
There are still a few deprecation warnings, but they don't cause the build to
fail.
Path remapping is now stable, so the buck config needs to change to use it
rather than the unstable command line option.
TODO:
- get aarch64 rust-crates-io build (can defer to a later update)
Reviewed By: Imxset21
Differential Revision: D7966091
fbshipit-source-id: 2e61e262c21eb01c852a36f49c6a6369cdaddcdb
Summary: this breaks the prod since we are not able to parse clienttelemetry being send by hg
Reviewed By: jsgf
Differential Revision: D7967361
fbshipit-source-id: e10eb52852ccb8c2878db01cd63b05cd7f901ae4
Summary:
@[1636530097:mung] reported that there was a bug wherein Diesel was incorrectly using MySQL's API by incorrectly calling `mysql_store_result` after `mysql_next_result` indicates that no more results are available. This behavior [appears to be intentional](705cdbe1fe/diesel/src/mysql/connection/raw.rs (L160)), but directly contradicts what the [documentation says is correct](https://dev.mysql.com/doc/refman/8.0/en/mysql-store-result.html). In MySQL 8.0, this results in the client getting an error for incorrect usage of the API.
I've added a local patch to [our fork of Diesel](https://github.com/kulshrax/diesel) on Github to [remove the extra call to `mysql_store_result`](7ae79c193e), and rebuilt the `rust-crates-io` tp2 project to use this version of Diesel. This diff updates the symlinks in fbcode to point to the updated tp2 project.
Reviewed By: lth
Differential Revision: D7961290
fbshipit-source-id: 804c599a5688cd54fb0fcc33688c608d068d7a7b
Summary: This allows us to put hgcli on the different host
Reviewed By: jsgf
Differential Revision: D7910980
fbshipit-source-id: 45bdd1ee42b54f8f1a425ed6950e785cdf85bfc3
Summary:
This is a (hopefully) short term hack to overcome the problem of overloading
Manifold.
Ideally manifold client has to adjust dynamically to the load. However
implementing it is
not trivial, so for now let's configure via config option.
Reviewed By: jsgf
Differential Revision: D7910979
fbshipit-source-id: c2dc32b592747732e7e6574e0fecf2d0aaef447e
Summary:
Curently hgcli can connect to Mononoke only if they are on the same host,
because unix domain socket are used. Also Mononoke has to use separate unix
domain socket for different repos.
The goal of this stack of diffs is to remove these limitations:
1 Make it possible to have hgcli and Mononoke server on different hosts
2 Make it possible to use one port/unix domain socket to connect to many repos.
This diff adds a separate thread that parses the Preamble, extracts the
reponame and sends request to a thread responsible for this repo.
Also hgcli now has a new cmd line option that specifies path to connect to.
Reviewed By: jsgf
Differential Revision: D7845156
fbshipit-source-id: a48bcfeec4755b2d5b3dfcf8e0383e6945178018
Summary: put_slice panics if not enough space. This diff fixes it
Reviewed By: jsgf
Differential Revision: D7876156
fbshipit-source-id: b1e2cc520e5c598588df9a8090fb70efb413b381
Summary:
Simple precaching. Reads all the manifests for a bookmark and up to
`commit_warmup_limit` of ancestors.
Warming up file content can be slow, so we don't do it now.
Reviewed By: jsgf
Differential Revision: D7863728
fbshipit-source-id: bed1508b01e4e002a399d00ea45faf8a8e228d0a
Summary: Replace all occurrences of the deprecated put_X::<BigEndian> methods with put_X_be in the files marked with T29077977, and remove any allow deprecation lines in the files and unused imports that might be introduced.
Reviewed By: jsgf
Differential Revision: D7928695
fbshipit-source-id: 8f16915f6b08aa55521637fff58a6cc27c13321a
Summary: extras use python's escaping functionality in mercurial, the '\r' case is happening in production data
Reviewed By: StanislavGlebik
Differential Revision: D7925149
fbshipit-source-id: 23c544b8a757c40a120b69d1ec1ad942c672de17
Summary:
This will make it easier to change the "real" bookmark type from AsciiString to
String if we decide to do that.
BookmarkPrefix is a separate type because we may want to change it from
AsciiString to String. Also we don't want to confuse a bookmark prefix with a
bookmark name.
Reviewed By: jsgf
Differential Revision: D7909992
fbshipit-source-id: 3d4d075c204ed5ef1114a743430982c2836bac04
Summary:
There are already many parameters, and I want to add a few more. Let's pass
RepoConfig directly
Reviewed By: jsgf
Differential Revision: D7861092
fbshipit-source-id: 76a6e4d2fbb06792fea50cee544f6fd62f670887
Summary: This will replace `RawCSBlob` and the current bincode serialization.
Reviewed By: StanislavGlebik
Differential Revision: D7869524
fbshipit-source-id: 1a2f5d159a20889b10bb6b235f48769da4a187c1
Summary:
Very similar to file envelopes.
Also add some missing `#[inline]` annotations for file envelopes.
Reviewed By: StanislavGlebik
Differential Revision: D7868445
fbshipit-source-id: 3fb0d87e087612f37c8d5d0a90065359c671ceb8
Summary:
We don't need to explicitly create timers since the environment has
one set up by default.
Reviewed By: StanislavGlebik
Differential Revision: D7873576
fbshipit-source-id: bfcdc27a46397bff0730f64ad4f3de3865c7cfa1
Summary:
Previously we wouldn't start processing next request until we are done with
this one.
It's easy to fix with asynchronize.
Reviewed By: jsgf
Differential Revision: D7835828
fbshipit-source-id: a9efef896f09815ca2f3e3659c814ad65edbb715
Summary:
Let's use the new feature in SendWrapper to use many io threads. That will help
us mitigate the high cpu usage issues we were having with blobstore requests.
Manifold blobstore now creates the io threads itself.
Reviewed By: kulshrax
Differential Revision: D7831420
fbshipit-source-id: ec9f3327347ca6bfbd23c482e69a6fee663b1da5
Summary: As with changesets and blobs, let's cache filenodes data
Reviewed By: jsgf
Differential Revision: D7831105
fbshipit-source-id: 334cb474f5cc3ef8dba0945d11273b2b3875e8ad
Summary:
Curently hgcli can connect to Mononoke only if they are on the same host, because unix domain socket are used. Also Mononoke has to use separate unix domain socket for different repos.
The goal of this stack of diffs is to remove these limitations:
1 Make it possible to have hgcli and Mononoke server on different hosts
2 Make it possible to use one port/unix domain socket to connect to many repos.
Check that that reponame from Preamble is the same as Mononoke reponame, and raise an error if it isn't.
Reviewed By: jsgf
Differential Revision: D7845157
fbshipit-source-id: 6567d5417ce58c4fba0364108592eed5bf0223c1
Summary:
Curently hgcli can connect to Mononoke only if they are on the same host, because unix domain socket are used. Also Mononoke has to use separate unix domain socket for different repos.
The goal of this stack of diffs is to remove these limitations:
1 Make it possible to have hgcli and Mononoke server on different hosts
2 Make it possible to use one port/unix domain socket to connect to many repos.
In this diff let's start sending Preamble as the first message from hgcli to Mononoke server.
For now Mononoke server will just ignore it. Later we'll use it to connect to the correct repo.
Reviewed By: jsgf
Differential Revision: D7845159
fbshipit-source-id: 3d2232028d931bfa7309e826b38084a87be1f85c
Summary:
Curently hgcli can connect to Mononoke only if they are on the same host,
because unix domain socket are used. Also Mononoke has to use separate unix
domain socket for different repos.
The goal of this stack of diffs is to remove these limitations:
1 Make it possible to have hgcli and Mononoke server on different hosts
2 Make it possible to use one port/unix domain socket to connect to many repos.
This diff adds a special message called Preamble. It will be the first message
sent by hgcli to Mononoke server. The most important information it contains is
reponame to tell Mononoke server what repo client it is interested in. Since
this is a crucial information, it is "required" i.e. Preamble will fail to
parse if reponame is not set. There can be other set in the `misc` field, for
example, user/host of a connecting user.
Currently Preamble is json encoded. We can change it in the future if we need
to.
Reviewed By: jsgf
Differential Revision: D7845158
fbshipit-source-id: 3cb98f0f7edcda8a0be2d5bfef1c7c4a18c8db6f
Summary:
Curently hgcli can connect to Mononoke only if they are on the same host,
because unix domain socket are used. Also Mononoke has to use separate unix
domain socket for different repos.
The goal of this stack of diffs is to remove these limitations:
1 Make it possible to have hgcli and Mononoke server on different hosts
2 Make it possible to use one port/unix domain socket to connect to many repos.
This diff makes a small refactoring to make next diff smaller. There should be
no change in the functionality
Reviewed By: jsgf
Differential Revision: D7845155
fbshipit-source-id: 723a974e3e648b3e3200139aea27fc34691b8983
Summary:
Curently hgcli can connect to Mononoke only if they are on the same host,
because unix domain socket are used. Also Mononoke has to use separate unix
domain socket for different repos.
The goal of this stack of diffs is to remove these limitations:
1 Make it possible to have hgcli and Mononoke server on different hosts
2 Make it possible to use one port/unix domain socket to connect to many repos.
It's not required. And in the next diff we are going to add new SshStream
value, that will contain non-hashable and non-copyable struct in it.
Reviewed By: jsgf
Differential Revision: D7845160
fbshipit-source-id: 75bdd1706b1bd2f1b16a9e8c793c346a093bdd54
Summary:
Specialized revsets to make pull faster.
Previous Union + Intersect combination was extremely slow because it fetched a
lot of stuff that wasn't used.
Reviewed By: farnz
Differential Revision: D7829394
fbshipit-source-id: c038f184c305e48e18b6fcb0f83bab9e9a42b098
Summary: The goal is to be able to read revlogs using Rust code and also parse and serialize them in Rust formats for debugging purposes
Reviewed By: farnz
Differential Revision: D7830358
fbshipit-source-id: 95e257a4482eca22b328b174bce3fceec1b47245
Summary:
The commits that are blobimported have out of order or simply incorrect lists of changed files.
Because we have to persists Changesets as is we are passing the untouched list of files here to be used by Changeset.
Reviewed By: farnz
Differential Revision: D7830310
fbshipit-source-id: 56adec2c317896decaa9176b3a6bfb0cab187ed0
Summary: Apparently p1 == p2 is a thing that happens in Mercurial, let's keep this logic in Mercurial-facing objects
Reviewed By: StanislavGlebik
Differential Revision: D7830183
fbshipit-source-id: eb473d57cb05f6553327bee2d5aeff9cf7d50eba
Summary: It can be useful to understand how often do we hit a db, write to blobstore, etc
Reviewed By: farnz
Differential Revision: D7806954
fbshipit-source-id: 9c85cbb11e184ae9f7751e05e961980b29b7568a
Summary:
If baserev == Some(idx), we changed it to None. Also we can have None if
baserev == -1 in mercurial. However these two cases are different. In the first
case it means that we have a literal chunk, not delta. In the second case it
means that we have a delta against empty string! So this is technically almost
the same, except that delta against empty string also have a 12 bytes prefix.
Previously this prefix was used as part of a revision data.
This diff fixes it.
Reviewed By: quark-zju
Differential Revision: D7815713
fbshipit-source-id: def2e54b2cc7379ba8f931ecf3f3c0c38d716058
Summary:
We want to be able to identify "interesting" sessions - add a Scuba
sample that tells us what wireprotocol commands were sent, and how long the
session lasted.
Reviewed By: jsgf
Differential Revision: D7813906
fbshipit-source-id: a9bd48996a60b41098243f6c815465cd33d1429c
Summary:
We want to be able to tie up our logs with a session identifier, so
that we can see if a given slow operation is expected (a complex session) or a
surprise. Give the client a logger that has a session ID tied in.
Reviewed By: jsgf
Differential Revision: D7813908
fbshipit-source-id: bc8a59046c19a0939fd044ca8b5b7ccc2956c5f0
Summary:
These have bitrotted, but I need a test case for a Scuba change. Fix
them.
Reviewed By: StanislavGlebik
Differential Revision: D7813907
fbshipit-source-id: e4e9b01a8a3c1de27f59c6d5ea695152df99d4ff
Summary: Useful utility that let's you f.e. fetch blob of data from manifold, decode it and show it to you
Reviewed By: jsgf
Differential Revision: D7779154
fbshipit-source-id: aaa4ae1d09b64f7f52c7942a51e8bb4ccc0cb700
Summary: The deltas are meant to be applied on raw filenode content that has incorporated copy/move information. Make it so
Reviewed By: jsgf
Differential Revision: D7777329
fbshipit-source-id: 23ce90269cdf5dbb8a16aab026c031c2af68fccb
Summary: The current behavior of delta::apply will panic Mononke server when the client sends malformed delta request. This change will instead propagate an Error explaining why the delta was malformed.
Reviewed By: jsgf
Differential Revision: D7775544
fbshipit-source-id: a64c27c7b9f13323b8be70eb8c2cdf315ce8f08d
Summary:
We'd like to move away from `RawNodeBlob` and `RawCSBlob` to data structures
serialized by Thrift. This is the first step to doing that.
The most important thing here is that it reuses file content IDs from native
Mononoke storage.
Reviewed By: jsgf
Differential Revision: D7771990
fbshipit-source-id: de4ee0b56aa6610caeff84b2235e19855df086cb
Summary: Want to get this in now before I forget.
Reviewed By: jsgf
Differential Revision: D7781551
fbshipit-source-id: a5e6fa062514d90113c96307f78e20fa05f770ea
