Summary: In this final diff for verify_integrity the data authorization data is being passed from hgcli (where the ssh connection is established) to Mononoke where it is further passed to verify_integrity script.
Reviewed By: StanislavGlebik
Differential Revision: D14387759
fbshipit-source-id: 2c0f9eef4128f5af0052276a1830ea2f449fb03a
Summary:
This additional data that is send to Mononoke will be used e.g. in verify_integrity hook.
This diff also contains change to the user_unixname, instead of passing env variable USER there we pass value produced by users crate. Since the user_unixname is not used anywhere now it is safe to do so.
Reviewed By: StanislavGlebik
Differential Revision: D14385280
fbshipit-source-id: 1e48c232fafbba5d5188c7d162e9ef21efd738f7
Summary:
We've had this problem for almost a year now, and I've finally made some
progress.
The problem was in tests failing randomly with error like
```
- remote: * pushrebase failed * (glob)
- remote: msg: "pushrebase failed Conflicts([PushrebaseConflict { left: MPath(\"1\"), right: MPath(\"1\") }])"
+ remote: Jan 25 08:46:24.067 ERRO Error in hgcli proxy, error: Connection reset by peer (os error 104), root_cause: Os {
+ remote: code: 104,
+ remote: kind: ConnectionReset,
+ remote: message: "Connection reset by peer"
remote: * backtrace* (glob)
```
or
```
remote: * pushrebase failed * (glob)
remote: msg: "pushrebase failed Conflicts([PushrebaseConflict { left: MPath(\"1\"), right: MPath(\"1\") }])"
+ remote: Jan 25 08:47:59.966 ERRO Error in hgcli proxy, error: Connection reset by peer (os error 104), root_cause: Os {
+ remote: code: 104,
+ remote: kind: ConnectionReset,
+ remote: message: "Connection reset by peer"
+ remote: }, backtrace:
```
note that the problem are slightly different. In the first case the actual error message is completely lost + we get unnecessary
ConnectionReset problem message. In the second case it's just `ConnectionReset`.
This diff fixes the problem of the lost error message (problem #1) and hides `ConnectionReset` problem (problem #2).
Problem #1 was due to a bug in streamfork. Before this diff if streamfork hit
an error, then it might have not sent already received input to one of the
outputs. This diff fixes it.
This diff just hides Problem #2. If we see a ConnectionReset then an error
won't be reported. That's a hack which should be fixed, but at the moment
a) The bug is not easily debuggable
b) The problem is not urgent and shouldn't cause problems
In some cases server actually sends Connection reset, but in that case
mercurial stil gives us self-explanatory message
```
abort: stream ended unexpectedly (got 0 bytes, expected 4
``
Reviewed By: lukaspiatkowski
Differential Revision: D13818558
fbshipit-source-id: 7a2cba8cd0fcef8211451df3dea558fe2d60fa60
Summary: Since this data is specific to TimedStream and not TimedFuture I split the Stats struct into FutureStats and StreamStats
Reviewed By: StanislavGlebik
Differential Revision: D9355421
fbshipit-source-id: cc2055706574756e2e53f3ccc57abfc50c3a02ba
Summary:
Those are the changes made to crates in tp2: P59806629
Several changes that were hard to split are here:
- update manifold client to use hyper 0.12.x
- update scribe client to use hyper 0.12.x
- update mononoke manifoldblob to use the updated manifold client
- update mononoke hgcli to use openssl 0.10.x and tokio-openssl
- update mononoke server to use openssl 0.10.x and tokio-openssl
- remove sendwrapper
Reviewed By: jsgf
Differential Revision: D8806931
fbshipit-source-id: 65412d483f77d8c0a0d5692c41c6516bb8f86046
Summary:
This is a series of patches which adds Cargo.toml files to all the crates and tries to build them. There is individual patch for each crate which tells whether that crate build successfully right now using cargo or not, and if not, reason behind that.
Following are the reasons why the crates don't build:
* failure_ext and netstring crates which are internal
* error related to tokio_io, there might be an patched version of tokio_io internally
* actix-web depends on httparse which uses nightly features
All the build is done using rustc version `rustc 1.27.0-dev`.
Pull Request resolved: https://github.com/facebookexperimental/mononoke/pull/7
Differential Revision: D8778746
Pulled By: jsgf
fbshipit-source-id: 927a7a20b1d5c9643869b26c0eab09e90048443e
Summary: by using futures rather than reactor::run calls we are able to add f.e. tracing and timeouts easily - in the next diff
Reviewed By: farnz
Differential Revision: D8750662
fbshipit-source-id: d928d903cf5631a275f5b319d075db9f1611d76b
Summary:
hgcli will start logging stuff as well and it will use the same session_uuid as the server.
This also includes logging the user and source hostname.
Reviewed By: farnz
Differential Revision: D8750663
fbshipit-source-id: 7ebc8b6c10b7560d985fd23e9e3f2645f3bd0a1c
Summary:
Use tls for connection between hgcli and Mononoke server always, even for
localhost connections[1]
The setup is similar to tls setup of Eden server.
[1] This is not necessary of course, but adding an option to bypass tls
connection may result in accidental use of it in prod. However if it turns out
to be too unusable, we can add such option in the future
Reviewed By: jsgf
Differential Revision: D8644299
fbshipit-source-id: 0898e30e33b718e13a766763479f3adf9323ffe7
Summary: Going to take a while to get to everything, but here's a start.
Reviewed By: Imxset21
Differential Revision: D8311107
fbshipit-source-id: ada1908b320a5277eda2587d7e8f26b13b952154
Summary:
We can hit a deadlock during getfiles requests. We first send all requests via
stdin, and then read the output via stdout. The problem is that any of the
buffers anywhere between hg and mononoke may fill up, and the whole process
deadlocks.
Unfortunately there is no good solution yet. For now let's make the buffer in
hgcli big enough to make sure it won't be a problem. Let's bump it to 400Mb.
Note that the buffer will be allocated on demand in the channel. So it won't
use too much memory.
Reviewed By: sunshowers
Differential Revision: D8195511
fbshipit-source-id: bd97509bd26f8cea7d41e44933e02bd0bc2c76c4
Summary:
This is a binary that reads json encoded requests from cmd line and sends them
to Mononoke. Usually requests will be sent via scribe from hg servers.
Then the data will be logged to scuba table `Mononoke Mercurial Comparison`
Also change hgcli to print errors to stderr
Reviewed By: farnz
Differential Revision: D8143781
fbshipit-source-id: 8d6c5511dfd2368f5d018825502c384f2ad89d4a
Summary: This allows us to put hgcli on the different host
Reviewed By: jsgf
Differential Revision: D7910980
fbshipit-source-id: 45bdd1ee42b54f8f1a425ed6950e785cdf85bfc3
Summary:
Curently hgcli can connect to Mononoke only if they are on the same host,
because unix domain socket are used. Also Mononoke has to use separate unix
domain socket for different repos.
The goal of this stack of diffs is to remove these limitations:
1 Make it possible to have hgcli and Mononoke server on different hosts
2 Make it possible to use one port/unix domain socket to connect to many repos.
This diff adds a separate thread that parses the Preamble, extracts the
reponame and sends request to a thread responsible for this repo.
Also hgcli now has a new cmd line option that specifies path to connect to.
Reviewed By: jsgf
Differential Revision: D7845156
fbshipit-source-id: a48bcfeec4755b2d5b3dfcf8e0383e6945178018
Summary:
Curently hgcli can connect to Mononoke only if they are on the same host, because unix domain socket are used. Also Mononoke has to use separate unix domain socket for different repos.
The goal of this stack of diffs is to remove these limitations:
1 Make it possible to have hgcli and Mononoke server on different hosts
2 Make it possible to use one port/unix domain socket to connect to many repos.
In this diff let's start sending Preamble as the first message from hgcli to Mononoke server.
For now Mononoke server will just ignore it. Later we'll use it to connect to the correct repo.
Reviewed By: jsgf
Differential Revision: D7845159
fbshipit-source-id: 3d2232028d931bfa7309e826b38084a87be1f85c
Summary:
The test wasn't waiting for the Mononoke server to be available. Also
improve the error message slightly.
Reviewed By: jsgf
Differential Revision: D6606102
fbshipit-source-id: d0fde39aef8b3423fd1a8996a01f12e8a9661597
Summary:
Realized that we were missing a few crates from the Tokio cleanup because those crates
didn't have `#![deny(warnings)]`.
This also caused a bunch of files to be rustfmted, which is fine.
Reviewed By: kulshrax
Differential Revision: D6024628
fbshipit-source-id: 55032d20f3676c92ef124d861e1edcd34126ab55
