Summary:
It happens quite often in practice, so we have to support it.
Luckily it's not that hard to implement
Reviewed By: jsgf
Differential Revision: D8076494
fbshipit-source-id: 460ba8b4bac121b84554e89b285d5ef78cff4838
Summary:
Log parameters to scuba for getfiles and gettreepack. These are the most
important wireproto methods.
Reviewed By: jsgf
Differential Revision: D8076492
fbshipit-source-id: 031bad732ee31d254a707ddd0e8c444d23c2fb0f
Summary: There is nothing we can do with it ATM
Reviewed By: StanislavGlebik
Differential Revision: D8090873
fbshipit-source-id: 232024938fe33583e328ad3231d9213dd6792601
Summary: csv crate has been updated to 1.0.0 which has API changes compared to the previous version.
Reviewed By: sunshowers
Differential Revision: D8014794
fbshipit-source-id: 8c5f148050c042df47a76fa2f8c7634794c69ccd
Summary: Once we have a lazy loading version, we'll want a way to cope with multiple accessors all trying to update the same `children` array with identical data; make it immutable in use, so that overwriting it multiple times is harmless
Reviewed By: jsgf
Differential Revision: D8014908
fbshipit-source-id: 9a2750fb1fca54601051fede1d9a37de8cfc2a74
Summary:
Instead of writing changesets one-by-one run mutiple of them at once.
The size `100` for the buffer is arbitrary, but it shouldn't matter much since we already have backpressure on the database writes.
Reviewed By: farnz
Differential Revision: D8057268
fbshipit-source-id: ca3766505395dcb6be6684323462f1bb23222435
Summary:
We had an asyncmemo bug when keys were, for example, Strings. If we insert
String::new("key"), and then try to remove String::new("key"), then these keys
may have different weights because Strings may have different capacity.
This is not the case for values though, because inserted and deleted value is
always the same object.
Let's not take key sizes into account at all. That means that the real asyncmemo memory usage will be
higher, however values are a great deal bigger than keys, so it sholdn't be an
issue.
Also we are planning to do it anyway, because we want to split
asyncmemo hashmap into in-flight futures and completed futures, and in-flight
futures won't take key sizes into account.
Reviewed By: jsgf
Differential Revision: D8040684
fbshipit-source-id: 332dbe5663182dd84ea291ca4b33b41995ac0166
Summary:
parking_lot's primitives are apparently lighter-weight than using
standard (pthreads) ones, and are more or less drop-in replacements.
Reviewed By: farnz
Differential Revision: D7661246
fbshipit-source-id: c7558a15971bf5b30c4dadb9437586832cdad3d4
Summary:
This tool will be installed on all devservers in our team. It should have
useful debugging commands. Currently it has a command to fetch an entry from
the blobstore, and a command to get content by path/commit hash.
Reviewed By: farnz
Differential Revision: D8028933
fbshipit-source-id: e0c37660a24e40dd9dc8f19d1789b2f25d99bfe6
Summary:
We're going to need to be able to edit memory manifests. Provide
remove and set operations, to match the Bonsai Changeset data structures
Reviewed By: StanislavGlebik
Differential Revision: D7620527
fbshipit-source-id: e85459c5dbfa8855267fd2cf6578c9fc39f223f8
Summary:
Removing entries can leave us with empty manifests. Rather than clean
that up mid-flow, simply skip empty manifests when saving
Reviewed By: StanislavGlebik
Differential Revision: D7620526
fbshipit-source-id: 2f0799eba305a5295eaad28cd4ad90c9de04306f
Summary:
To do Bonsai Changesets, we're going to need to perform surgery on
manifests. Provide a mechanism to get the Tree nodes only into memory, and
write them back out, so that we can do the surgery the easy way.
Reviewed By: StanislavGlebik
Differential Revision: D7557271
fbshipit-source-id: 7afdc3ef464fc042eb758af863ade8938c4e9fc5
Summary:
Previously, we assumed that all content hashes came from Mercurial;
this is not going to remain true, as we will want to be able to upload manifests
that have been synthesised from Bonsai Changesets. Turn the previous boolean
into a tri-state, and fix up all callers to get the behaviour they expect.
Reviewed By: StanislavGlebik
Differential Revision: D8014911
fbshipit-source-id: 9156b9fab4542ceb269626ad005e1b28392b5329
Summary: Parsing and reading revlogs is cpu intensive, thus let's use cpupool for it
Reviewed By: StanislavGlebik
Differential Revision: D7926174
fbshipit-source-id: 7f023088941e1ad118a683da972f87607e0bfec4
Summary: printing every CS is too verbose, but we still want to see progress in non-debug mode
Reviewed By: kulshrax
Differential Revision: D7925747
fbshipit-source-id: c3ed92ef8c8fbf7714779a2bf011d31c94aefa37
Summary:
Rust 1.26 adds many new language features. In particular `impl Trait` is now
stable, so we no longer need `conservative_impl_trait`.
There also seems to have been changed in the (unstable) TryFrom with respect to
usize, and the behaviour of the never type `!`.
There are still a few deprecation warnings, but they don't cause the build to
fail.
Path remapping is now stable, so the buck config needs to change to use it
rather than the unstable command line option.
TODO:
- get aarch64 rust-crates-io build (can defer to a later update)
Reviewed By: Imxset21
Differential Revision: D7966091
fbshipit-source-id: 2e61e262c21eb01c852a36f49c6a6369cdaddcdb
Summary: this breaks the prod since we are not able to parse clienttelemetry being send by hg
Reviewed By: jsgf
Differential Revision: D7967361
fbshipit-source-id: e10eb52852ccb8c2878db01cd63b05cd7f901ae4
Summary:
@[1636530097:mung] reported that there was a bug wherein Diesel was incorrectly using MySQL's API by incorrectly calling `mysql_store_result` after `mysql_next_result` indicates that no more results are available. This behavior [appears to be intentional](705cdbe1fe/diesel/src/mysql/connection/raw.rs (L160)), but directly contradicts what the [documentation says is correct](https://dev.mysql.com/doc/refman/8.0/en/mysql-store-result.html). In MySQL 8.0, this results in the client getting an error for incorrect usage of the API.
I've added a local patch to [our fork of Diesel](https://github.com/kulshrax/diesel) on Github to [remove the extra call to `mysql_store_result`](7ae79c193e), and rebuilt the `rust-crates-io` tp2 project to use this version of Diesel. This diff updates the symlinks in fbcode to point to the updated tp2 project.
Reviewed By: lth
Differential Revision: D7961290
fbshipit-source-id: 804c599a5688cd54fb0fcc33688c608d068d7a7b
Summary: This allows us to put hgcli on the different host
Reviewed By: jsgf
Differential Revision: D7910980
fbshipit-source-id: 45bdd1ee42b54f8f1a425ed6950e785cdf85bfc3
Summary:
This is a (hopefully) short term hack to overcome the problem of overloading
Manifold.
Ideally manifold client has to adjust dynamically to the load. However
implementing it is
not trivial, so for now let's configure via config option.
Reviewed By: jsgf
Differential Revision: D7910979
fbshipit-source-id: c2dc32b592747732e7e6574e0fecf2d0aaef447e
Summary:
Curently hgcli can connect to Mononoke only if they are on the same host,
because unix domain socket are used. Also Mononoke has to use separate unix
domain socket for different repos.
The goal of this stack of diffs is to remove these limitations:
1 Make it possible to have hgcli and Mononoke server on different hosts
2 Make it possible to use one port/unix domain socket to connect to many repos.
This diff adds a separate thread that parses the Preamble, extracts the
reponame and sends request to a thread responsible for this repo.
Also hgcli now has a new cmd line option that specifies path to connect to.
Reviewed By: jsgf
Differential Revision: D7845156
fbshipit-source-id: a48bcfeec4755b2d5b3dfcf8e0383e6945178018
Summary: put_slice panics if not enough space. This diff fixes it
Reviewed By: jsgf
Differential Revision: D7876156
fbshipit-source-id: b1e2cc520e5c598588df9a8090fb70efb413b381
Summary:
Simple precaching. Reads all the manifests for a bookmark and up to
`commit_warmup_limit` of ancestors.
Warming up file content can be slow, so we don't do it now.
Reviewed By: jsgf
Differential Revision: D7863728
fbshipit-source-id: bed1508b01e4e002a399d00ea45faf8a8e228d0a
Summary: Replace all occurrences of the deprecated put_X::<BigEndian> methods with put_X_be in the files marked with T29077977, and remove any allow deprecation lines in the files and unused imports that might be introduced.
Reviewed By: jsgf
Differential Revision: D7928695
fbshipit-source-id: 8f16915f6b08aa55521637fff58a6cc27c13321a
Summary: extras use python's escaping functionality in mercurial, the '\r' case is happening in production data
Reviewed By: StanislavGlebik
Differential Revision: D7925149
fbshipit-source-id: 23c544b8a757c40a120b69d1ec1ad942c672de17
Summary:
This will make it easier to change the "real" bookmark type from AsciiString to
String if we decide to do that.
BookmarkPrefix is a separate type because we may want to change it from
AsciiString to String. Also we don't want to confuse a bookmark prefix with a
bookmark name.
Reviewed By: jsgf
Differential Revision: D7909992
fbshipit-source-id: 3d4d075c204ed5ef1114a743430982c2836bac04
Summary:
There are already many parameters, and I want to add a few more. Let's pass
RepoConfig directly
Reviewed By: jsgf
Differential Revision: D7861092
fbshipit-source-id: 76a6e4d2fbb06792fea50cee544f6fd62f670887
Summary: This will replace `RawCSBlob` and the current bincode serialization.
Reviewed By: StanislavGlebik
Differential Revision: D7869524
fbshipit-source-id: 1a2f5d159a20889b10bb6b235f48769da4a187c1
Summary:
Very similar to file envelopes.
Also add some missing `#[inline]` annotations for file envelopes.
Reviewed By: StanislavGlebik
Differential Revision: D7868445
fbshipit-source-id: 3fb0d87e087612f37c8d5d0a90065359c671ceb8
Summary:
We don't need to explicitly create timers since the environment has
one set up by default.
Reviewed By: StanislavGlebik
Differential Revision: D7873576
fbshipit-source-id: bfcdc27a46397bff0730f64ad4f3de3865c7cfa1
Summary:
Previously we wouldn't start processing next request until we are done with
this one.
It's easy to fix with asynchronize.
Reviewed By: jsgf
Differential Revision: D7835828
fbshipit-source-id: a9efef896f09815ca2f3e3659c814ad65edbb715
Summary:
Let's use the new feature in SendWrapper to use many io threads. That will help
us mitigate the high cpu usage issues we were having with blobstore requests.
Manifold blobstore now creates the io threads itself.
Reviewed By: kulshrax
Differential Revision: D7831420
fbshipit-source-id: ec9f3327347ca6bfbd23c482e69a6fee663b1da5
Summary: As with changesets and blobs, let's cache filenodes data
Reviewed By: jsgf
Differential Revision: D7831105
fbshipit-source-id: 334cb474f5cc3ef8dba0945d11273b2b3875e8ad
Summary:
Curently hgcli can connect to Mononoke only if they are on the same host, because unix domain socket are used. Also Mononoke has to use separate unix domain socket for different repos.
The goal of this stack of diffs is to remove these limitations:
1 Make it possible to have hgcli and Mononoke server on different hosts
2 Make it possible to use one port/unix domain socket to connect to many repos.
Check that that reponame from Preamble is the same as Mononoke reponame, and raise an error if it isn't.
Reviewed By: jsgf
Differential Revision: D7845157
fbshipit-source-id: 6567d5417ce58c4fba0364108592eed5bf0223c1
Summary:
Curently hgcli can connect to Mononoke only if they are on the same host, because unix domain socket are used. Also Mononoke has to use separate unix domain socket for different repos.
The goal of this stack of diffs is to remove these limitations:
1 Make it possible to have hgcli and Mononoke server on different hosts
2 Make it possible to use one port/unix domain socket to connect to many repos.
In this diff let's start sending Preamble as the first message from hgcli to Mononoke server.
For now Mononoke server will just ignore it. Later we'll use it to connect to the correct repo.
Reviewed By: jsgf
Differential Revision: D7845159
fbshipit-source-id: 3d2232028d931bfa7309e826b38084a87be1f85c
Summary:
Curently hgcli can connect to Mononoke only if they are on the same host,
because unix domain socket are used. Also Mononoke has to use separate unix
domain socket for different repos.
The goal of this stack of diffs is to remove these limitations:
1 Make it possible to have hgcli and Mononoke server on different hosts
2 Make it possible to use one port/unix domain socket to connect to many repos.
This diff adds a special message called Preamble. It will be the first message
sent by hgcli to Mononoke server. The most important information it contains is
reponame to tell Mononoke server what repo client it is interested in. Since
this is a crucial information, it is "required" i.e. Preamble will fail to
parse if reponame is not set. There can be other set in the `misc` field, for
example, user/host of a connecting user.
Currently Preamble is json encoded. We can change it in the future if we need
to.
Reviewed By: jsgf
Differential Revision: D7845158
fbshipit-source-id: 3cb98f0f7edcda8a0be2d5bfef1c7c4a18c8db6f
Summary:
Curently hgcli can connect to Mononoke only if they are on the same host,
because unix domain socket are used. Also Mononoke has to use separate unix
domain socket for different repos.
The goal of this stack of diffs is to remove these limitations:
1 Make it possible to have hgcli and Mononoke server on different hosts
2 Make it possible to use one port/unix domain socket to connect to many repos.
This diff makes a small refactoring to make next diff smaller. There should be
no change in the functionality
Reviewed By: jsgf
Differential Revision: D7845155
fbshipit-source-id: 723a974e3e648b3e3200139aea27fc34691b8983
Summary:
Curently hgcli can connect to Mononoke only if they are on the same host,
because unix domain socket are used. Also Mononoke has to use separate unix
domain socket for different repos.
The goal of this stack of diffs is to remove these limitations:
1 Make it possible to have hgcli and Mononoke server on different hosts
2 Make it possible to use one port/unix domain socket to connect to many repos.
It's not required. And in the next diff we are going to add new SshStream
value, that will contain non-hashable and non-copyable struct in it.
Reviewed By: jsgf
Differential Revision: D7845160
fbshipit-source-id: 75bdd1706b1bd2f1b16a9e8c793c346a093bdd54
Summary:
Specialized revsets to make pull faster.
Previous Union + Intersect combination was extremely slow because it fetched a
lot of stuff that wasn't used.
Reviewed By: farnz
Differential Revision: D7829394
fbshipit-source-id: c038f184c305e48e18b6fcb0f83bab9e9a42b098
Summary: The goal is to be able to read revlogs using Rust code and also parse and serialize them in Rust formats for debugging purposes
Reviewed By: farnz
Differential Revision: D7830358
fbshipit-source-id: 95e257a4482eca22b328b174bce3fceec1b47245
Summary:
The commits that are blobimported have out of order or simply incorrect lists of changed files.
Because we have to persists Changesets as is we are passing the untouched list of files here to be used by Changeset.
Reviewed By: farnz
Differential Revision: D7830310
fbshipit-source-id: 56adec2c317896decaa9176b3a6bfb0cab187ed0
