Commit Graph

32 Commits

Author SHA1 Message Date
Stanislau Hlebik
b35bd5dd9d mononoke: pushkey part parsing
Summary:
Pushkey part is used to send bookmark updates from hg client to the server.
This diff does all the wireproto parsing, but doesn't actually apply bookmark
updates on the server.

Also this diff "implements" branchmap method. We have no plans to support it,
but currently remotenames extension calls it. So this diff adds a fake
implementation that always returns empty response.

Reviewed By: farnz

Differential Revision: D7150973

fbshipit-source-id: 6889c02a1105127b1805ef1fafa6fbe9c2d57e7d
2018-03-16 04:21:32 -07:00
Stanislau Hlebik
11af232471 mononoke: send replychangegroup to the client
Summary:
Return a reply to a client so that it doesn't fail.
Reply consists of just one replychangegroup that tells a client that the push
has succeeded (even though currently it wasn't).

resolve() function returns future of Bytes instead of a stream. It may be
suboptimal, but should be fine for now, and it's already used by a few
wireproto methods.

Reviewed By: lukaspiatkowski

Differential Revision: D7010578

fbshipit-source-id: 9b5425b912c640d4e2bac957a02e9881813b8871
2018-02-21 11:07:19 -08:00
Stanislau Hlebik
06783fc66c mononoke: change bundle2_resolver::resolve signature
Summary:
Now resolve() function accepts stream of Bundle2Item and it doesn't return
remainder. Remainder is parsed in hgproto. This makes unbundle() similar to
other hgproto methods.

Note that resolve() returns future of () - this is temporary and it'll be
changed in the next diff.

Reviewed By: jsgf

Differential Revision: D7010577

fbshipit-source-id: f4bde495d0ab8785476559f65b0165dbd09e4ba5
2018-02-19 05:26:12 -08:00
Lukas Piatkowski
52470ce76c mercurial-bundles: hardcode 'static lifetime in Bundle2Stream
Summary: The only real use case of Bundle2Stream requires it to be 'static, so the is no real point of preserving 'a.

Reviewed By: StanislavGlebik

Differential Revision: D6965528

fbshipit-source-id: 3d5c9c74c1d17980befd3742c54cdde9279aa62c
2018-02-15 09:52:45 -08:00
Stanislau Hlebik
22a09370b7 mononoke: basic lookup implementation
Summary:
Wireproto method to lookup if a revision/bookmark exists.
Currently supports only full hashes, later we'll add support for bookmarks and
node prefixes.

Reviewed By: farnz

Differential Revision: D6948358

fbshipit-source-id: af92a892eae8f787053447e601ebf95dfd638fa4
2018-02-13 10:21:43 -08:00
Stanislau Hlebik
0ad9a311ff mononoke: getfiles arguments parsing
Summary:
getfiles is a remotefilelog method to fetch file content. Mercurial client
calls it on-demand, usually during `hg update` command. So normal `hg pull`
with remotefilelog doesn't send filelogs back to the client.

This diff implements getfiles arguments parsing. The code is tricky, so I've
added lots of comments to explain what's going on.

Reviewed By: farnz

Differential Revision: D6913178

fbshipit-source-id: 0248993c3ac487956e0ff547996d51b75cdaee96
2018-02-13 08:54:31 -08:00
Stanislau Hlebik
04c30c3e46 mononoke: gettreepack command and param parsing
Summary:
Everything needed for gettreepack method to work except for the gettreepack
implementation

Reviewed By: lukaspiatkowski

Differential Revision: D6845034

fbshipit-source-id: b675a2c38c7f6204fac5a6a0885caef246ba7987
2018-02-07 07:53:48 -08:00
Lukas Piatkowski
7d74c4c0d9 hgproto: return stream of responses to get infinite back-and-forths in handler
Summary: This change let's the handler return multiple responses to the client that is required for commands like `unbundle` to work

Reviewed By: jsgf

Differential Revision: D6820845

fbshipit-source-id: 5f17c2a6100aa989f35d468c188322da4a921059
2018-02-06 06:22:20 -08:00
Lukas Piatkowski
0572acd0f2 hgproto: Dechunker for decoding streaming arguments of a request
Summary: Dechunker parses a hgproto stream of bytes by taking the chunks and letting the code to read them via BufRead API

Reviewed By: jsgf

Differential Revision: D6619521

fbshipit-source-id: 26dfab554a8e4c79079f4c4666c272ec2924c7d2
2018-02-06 06:22:19 -08:00
Stanislau Hlebik
d317ac60fc mononoke: fix error message
Reviewed By: farnz

Differential Revision: D6871203

fbshipit-source-id: a3700d898bcaecf3ff94993fc5c84089b115bef0
2018-02-06 02:20:39 -08:00
Lukas Piatkowski
2e0a2819d2 hgproto: return '0\n' as response for unbundle request
Summary: This is what the client expexts to receive before streaming can happen

Reviewed By: farnz

Differential Revision: D6833721

fbshipit-source-id: 04a66415638187108366109b0d9062bd8481a628
2018-02-02 10:13:20 -08:00
Lukas Piatkowski
ab6de6b0cd hgproto: make the handler return future of response and future of unprocessed data
Summary:
The client sends us a request that we parse and return response for it. As simple as that.

When the client sends `unbundle` request we need to return an acknowledgment that we are ready to read a bundle2 stream from it.
This means that we first parse the request, then send the response, then parse the stream.

This will be changed in future diffs so that we might:
Parse request from client -> [send acknowledgments -> parse data from client]* -> send response;

Reviewed By: farnz

Differential Revision: D6833723

fbshipit-source-id: 0c13b1e3253bd664bc565eacb5b58a9fc53d4efa
2018-02-02 10:13:20 -08:00
Jeremy Fitzhardinge
9df09a500c mononoke/hgproto: remove dep on tokio-proto
Summary: It wasn't needed

Reviewed By: Imxset21

Differential Revision: D6862155

fbshipit-source-id: 21a0d1202e32caba96fb7d6e0923bf2f99f028ea
2018-01-31 13:36:45 -08:00
Stanislau Hlebik
0ba15b96a5 mononoke: fix lookup command param parsing
Summary:
Previously parsing code assumed that lookup param is an ident string. This is
incorrect, because ident string accepts only strings that consist only of
digits and small letters and start with a letter. Since a hash can be a lookup
param, this param may start with non-letter. More importantly, any bookmark can
be a lookup parameter. That means that this param can have spaces and
punctuation marks.

Let's use simpler parsing that just parses the whole string.

Reviewed By: jsgf

Differential Revision: D6758916

fbshipit-source-id: de2dd144cebfe88c751e8110ef6e3a71d3769800
2018-01-21 07:37:30 -08:00
Siddharth Agarwal
9697c1133e hgproto: accept empty hashlists and pairlists
Summary:
`complete!` had to be wrapped around both the separator and the input parser.
`separated_list_complete!` does that automatically, which is nice.

Reviewed By: jsgf

Differential Revision: D6758479

fbshipit-source-id: a2fe94a422a1b4a1c09bbfa915cde19d968d3273
2018-01-19 13:51:03 -08:00
Lukas Piatkowski
70244ff1d6 hgproto: nearly end-to-end support for handling protocol in streaming fashion
Reviewed By: jsgf

Differential Revision: D6494647

fbshipit-source-id: e685b33a35f49365d2cfa89491b1566cfcc75118
2018-01-15 10:36:32 -08:00
Lukas Piatkowski
8244a3c4b5 mercurial_bundle: return remaining data as last item of stream
Reviewed By: jsgf

Differential Revision: D6601718

fbshipit-source-id: 34019701bda3e187ce77a85ef04fef5d726d2286
2018-01-15 10:36:31 -08:00
Lukas Piatkowski
533cad062f futures-ext: remove FramedStream and replace it's usages with tokio_io::codec::Framed
Summary: The Framed and FramedParts duet fits following diffs a bit better than FramedStream and ReadLeadingBuffer.

Reviewed By: jsgf

Differential Revision: D6567554

fbshipit-source-id: 88d117ad9e8227f9de278037b333da7ffc4fdf1f
2018-01-15 10:36:31 -08:00
Lukas Piatkowski
f59d6a1c8b async-compression: make sure decompression is framed and won't overread it's input
Summary:
- Added test for checking if decompressing isn't overreading it's input
  - As a result removed ZSTD decompression which is overreading input
- Changes in decompression code propagated usage of BufRead in bundle2 parsing code
  - As a result a bug in OuterDecoder was found where the buf was not consumed if the header value is "0"

Reviewed By: jsgf

Differential Revision: D6557440

fbshipit-source-id: 89a9f4c8790017c5b86d28d467e45f687d7323f6
2018-01-15 10:36:31 -08:00
Simon Farnsworth
ebafde00b0 Remove Repo trait completely
Summary:
We're never going to serve RevlogRepo in production, and we're down to
a single BlobRepo type that will have different backing stores. Remove the
unused trait, and use BlobRepo everywhere bar blobimport and repo_config
(because we previously hardcoded revlog here - we want to change to a BlobRepo
once blobimport is full-fidelity).

Reviewed By: jsgf

Differential Revision: D6596164

fbshipit-source-id: ba6e76e78c495720792cbe77ae6037f7802ec126
2018-01-15 06:37:27 -08:00
Jeremy Fitzhardinge
ac31713c84 rust: failure cleanup pass
Summary:
Don't use failure's bail!() and ensure!() macros.

Instead, failure_ext provides:
- bail_err!(err) - Converts its single parameter to the expected error and returns; ie `return Err(From::from(err));`
- bail_msg!(fmt, ...) - takes format string parameters and returns a `failure::err_msg()` error
- ensure_err!(), ensure_msg!() - corresponding changes

Also:
- remove all stray references to error-chain
- remove direct references to failure_derive (it's reexported via failure and failure_ext)
- replace uses of `Err(foo)?;` with `bail_err!()` (since `bail_err` unconditionally returns, but `Err(x)?` does not in principle, which can affect type inference)

Reviewed By: kulshrax

Differential Revision: D6507717

fbshipit-source-id: 635fb6f8c96d185b195dff171ea9c8db9e83af10
2017-12-07 14:10:17 -08:00
Lukas Piatkowski
d75a7f1a18 hgproto: make a clear distinction between Batch and other requests
Summary: This diff also moves parsing of Batch subcommands inside sshproto where it belongs

Reviewed By: farnz

Differential Revision: D6492001

fbshipit-source-id: a31864f50e9468f360f9a30fc75fc2cfdee56169
2017-12-07 05:32:24 -08:00
Jeremy Fitzhardinge
dc5e78c1c1 rust: mass convert scm/mononoke/... to use failure
Summary:
Convert scm/mononoke to use failure, and update common/rust crates it depends on as well.

What it looks like is a lot of deleted code...

General strategy:
- common/rust/failure_ext adds some things that are in git failure that aren't yet in crates.io (`bail!` and `ensure!`, `Result<T, Error>`)
- everything returns `Result<T, failure::Error>`
- crates with real error get an error type, with a derived Fail implementation
  - replicate error-chain by defining an `enum ErrorKind` where the fields match the declared errors in the error! macro
- crates with dummy error-chain (no local errors) lose it
- `.chain_err()` -> `.context()` or `.with_context()`

So far the only place I've needed to extract an error is in a unit test.
Having a single unified error type has simplified a lot of things, and removed a lot of error type parameters, error conversion, etc, etc.

Reviewed By: sid0

Differential Revision: D6446584

fbshipit-source-id: 744640ca2997d4a85513c4519017f2e2e78a73f5
2017-12-05 18:11:13 -08:00
Simon Farnsworth
b38dd81782 Make known protocol check that the commit is fully present
Summary: `changeset_exists` only checks that the changeset is in the blob store. Once we start accepting pushes, we'll also want to check that the changeset is reachable from a head. Use revsets to check this. Once we have a notion of "completeness" for changesets, we'll switch `known` to use that instead - this code is still useful as a way to go from `changeset_exists` to `changeset_complete` in bulk.

Reviewed By: jsgf

Differential Revision: D6205695

fbshipit-source-id: 7e3b5c30bc5e459feb95a20913d8a04f3fda7469
2017-12-04 11:54:53 -08:00
Lukas Piatkowski
e55e4a3a2d hgproto: introduce stream based handling of input and output of hg protocol
Summary: This is the first step that establishes a stream based API, later diffs will propagate this idea downstream, so that we fully support streaming in f.e. unbundle bundle2

Reviewed By: jsgf

Differential Revision: D6239508

fbshipit-source-id: 43afba4f640586b728cb4cb85d14226a677ee58b
2017-11-08 12:30:47 -08:00
Jeremy Fitzhardinge
04276f43b4 mononoke: hgproto: capture unbundle stream as Bytes
Summary:
Parse the unbundle command as an `unbundle` command with its `heads`
parameter, followed by a bundle2 stream which is parsed in order to capture it
as bytes, as a simulation of an actual stream.

Reviewed By: sid0

Differential Revision: D6209818

fbshipit-source-id: 9bc454319350e2047160347964740f3a3d2f592f
2017-11-01 18:51:21 -07:00
Jeremy Fitzhardinge
6ba05b01a8 mononoke: hgproto: apply rustfmt
Reviewed By: farnz

Differential Revision: D6203916

fbshipit-source-id: 0f085672b7c584baa08563cc0365c3ef6fbecdfb
2017-11-01 18:51:21 -07:00
Stanislau Hlebik
a231885209 mononoke: fix param parsing for unbundle
Summary:
Even though the parameter is called `heads`, it's not necessarily heads.
It can be:
1) hex-encoded word "force" - "666f726365"
2) Hex-encoded list of heads
3) hex encoded word "hashed" and a hash of heads' hashes.

Proof: https://www.mercurial-scm.org/repo/hg/file/tip/mercurial/wireproto.py#l362

Previously we just parsed list of hashes, and it wasn't correct. This diff fixes it

Reviewed By: jsgf

Differential Revision: D6110315

fbshipit-source-id: 57ca12d13e3b719803b20ae1718d5f4309b806f5
2017-10-26 07:20:12 -07:00
Yu Liu
c98955694d remove uses of deprecated Tokio APIs from fbcode/scm/mononoke/hgproto
Summary:
Removes usage of BoxFuture and BoxStream from futures crate and instead use
it from internal futures_ext lib.

Reviewed By: sid0

Differential Revision: D6018186

fbshipit-source-id: 4d4b17fad1d9adc8c9de93ce731f6dfebd66e3b5
2017-10-10 08:06:12 -07:00
Siddharth Agarwal
a74542da60 update and pull in uuid + subprocess
Summary:
Finally got an update working by removing the `mysql_async` crate.

Some notes:

* The `mysql_async` crate was responsible in this case: see https://github.com/rust-lang/cargo/issues/4066#issuecomment-328210204 for why.
* tokio/futures deprecated a bunch of stuff. I've filed a TODO for now.
* We finally pulled in error-chain 0.11, which has a bunch of nice improvements.

Reviewed By: kulshrax

Differential Revision: D5798282

fbshipit-source-id: a38a7b17ee0205428e2ea63334722aa408582493
2017-09-08 17:05:38 -07:00
Jeremy Fitzhardinge
025c4ace72 mononoke: hgproto: generic Repo error
Summary: Add placeholder error in hgproto for errors from Repo operations. This is intended to be chained onto the real error.

Reviewed By: sid0

Differential Revision: D5540669

fbshipit-source-id: dedf12c9c749c40ba77444fb0adb30c8ddc868ed
2017-08-02 17:43:24 -07:00
facebook-github-bot
2b6af6b941 Initial commit
fbshipit-source-id: f75baa4ff6aa71973f677b752d7aba582cf4927f
2017-07-27 18:00:19 -07:00