Commit Graph

28 Commits

Author SHA1 Message Date
Stefan Filip
09aa2f900f bindings: use RepoPath in edenapi
Summary: migration

Differential Revision: D14945335

fbshipit-source-id: 0aa90bee100fdae8755e7abfa6ac614f5bcaf266
2019-04-16 15:34:31 -07:00
Stefan Filip
7e2b3c256f types: rename Key::new to Key::from_name_slice
Summary:
We should update the builder for Key to take a repo path. We could build
the key directly using the default struct constructor but representing
the two constructors as functions is more clear.

Reviewed By: quark-zju

Differential Revision: D14877543

fbshipit-source-id: 328906521cdbad535e28df22fea82f21e8b5410a
2019-04-14 19:56:50 -07:00
Arun Kulshreshtha
de2605bf4d edenapi: make data and history batch sizes separately configurable
Summary: Make the batch size of data and history requests independently configurable, since data responses are typically much larger than history responses (since the former contains actual file data whereas the latter is only metadata).

Differential Revision: D14859686

fbshipit-source-id: c87c31f3e6611a55ae712e7f0ed9bb392d31a579
2019-04-09 17:00:01 -07:00
Arun Kulshreshtha
9fdd71e4df edenapi: use curl multi interface
Summary: Use the curl multi interface to fetch multiple batches of files or history entries concurrently.

Differential Revision: D14718547

fbshipit-source-id: c5a740c7e9106b719e825540f8182be31a72bae7
2019-04-09 14:59:44 -07:00
Arun Kulshreshtha
6705e2d120 bindings: allow choice between edenapi backends
Summary: Allow users to configure which HTTP client backend to use for the Eden API via the `edenapi.backend` config option. Valid options are `curl` and `hyper`, with `curl` being the default.

Reviewed By: quark-zju

Differential Revision: D14657871

fbshipit-source-id: 7a9972d2380fbbd5ed62d1accae764dc03ca4c29
2019-04-05 17:34:14 -07:00
Arun Kulshreshtha
6c8c87dea1 edenapi: add curl-based client
Summary:
Add a new Eden API client based on libcurl (via the rust-curl crate). This should help us work around issues with Hyper.

This implementation is based on curl's "easy" API, and is intentionally naive. I intend to update it to use curl's "multi" API to send several concurrent HTTP requests per operation in a later diff.

Differential Revision: D14656756

fbshipit-source-id: 1f71074506844104f0f3237023b38317a7f41979
2019-04-05 17:34:14 -07:00
Arun Kulshreshtha
d5c9b17b05 bindings: turn off non_camel_case_types warnings
Summary: We're intentionally exporting Python classes with names that match Mercurials convention of `allonewordlowercase`, so we should turn off this lint.

Differential Revision: D14760996

fbshipit-source-id: cf79e8ebb6a01bd7bcfb062754cb034d94ac493b
2019-04-03 18:27:28 -07:00
Arun Kulshreshtha
1472ff1efa edenapi: refactor Builder into Config
Summary:
Rather than having a `Builder` struct that knows how to build just one kind of Eden API client, let's have a common `Config` type that can be potentially passed to the constructor of several different client implementations.

This will allow the same config code to be re-used across different client types, as seen later in this stack.

Differential Revision: D14656757

fbshipit-source-id: 883ffd2dc0302ebe08960f079c113e2d0da2d2ca
2019-04-01 20:15:38 -07:00
Mark Thomas
6d84ff8825 mutation: create mutation store entries for local commits
Summary:
Computing mutation entries for all local commits is expensive when there are
lots of local draft commits.  Ideally we would have an indexed changelog that
would make these lookups fast, but until we have that, put entries in the
mutation store for these commits to take advantage of the fast lookup there.

Reviewed By: quark-zju

Differential Revision: D14566782

fbshipit-source-id: cc3a05715337a510a65d8ff436c59d16d0f0447e
2019-03-27 04:49:12 -07:00
Mark Thomas
6ce749a3e1 infinitepush: exchange mutation information via infinitepush bundles
Summary:
Include mutation records for all predecessors of the pushed commits in
infinitepush bundles.

When received from infinitepush, store these additional records in the local
store.  This allows us to bridge any gaps in mutation that are omitted from the
local commits when they are received from infinitepush.

Reviewed By: DurhamG, quark-zju

Differential Revision: D12980777

fbshipit-source-id: b1535ca29c0fca3e6cb0f563d78c4c60d4aef58e
2019-03-08 03:05:57 -08:00
Jun Wu
426a80a117 zstd: back out "make Python APIs zero-copy"
Summary:
Original commit changeset: af43d4cce555

D14338313 didn't trigger the main `.t` tests. A lot of things actually
depend on `zstd` APIs returning `bytes`. Therefore back out the change.

Reviewed By: DurhamG

Differential Revision: D14372351

fbshipit-source-id: d8daa7d1d2a49d9d0c4d48de22ed0567d1d953a7
2019-03-07 13:30:10 -08:00
Jun Wu
932bb1492c zstd: make Python APIs zero-copy
Summary:
Make them zero-copy to reduce overhead. A side effect is the types become
`bytearray`, instead of `bytes`. It seems fine for zstd use-case.

Reviewed By: ikostia

Differential Revision: D14338313

fbshipit-source-id: af43d4cce5559fe877373737a71e1e1678a17ca7
2019-03-06 21:35:10 -08:00
Arun Kulshreshtha
b855ea2fbc fileserverclient: add ability to request packs over http
Summary: Allow file data/history packs to be fetched via HTTP when the Eden API is enabled.

Reviewed By: quark-zju

Differential Revision: D14257368

fbshipit-source-id: 8b6823a57a6fdef546a596df20387b3fc1ccdd4a
2019-02-28 17:10:47 -08:00
Arun Kulshreshtha
5f578e136c remotefilelog: add debuggethistory command
Summary: This diff adds a new `hg debuggethistory` command that takes filenode/path pairs from stdin, fetches the history of the files from the API server, and writes the results to a historypack in the hg cache.

Reviewed By: quark-zju

Differential Revision: D14248082

fbshipit-source-id: 8014a758abd3a578ea213d8d3177812629b2fd51
2019-02-28 15:42:17 -08:00
Arun Kulshreshtha
55171a856b remotefilelog: attach edenapi client to repo object
Summary: The Eden API client in Mercurial should be a singleton. This diff assigns the client to `repo.edenapi` so that it is accessible throughout the code.

Reviewed By: quark-zju

Differential Revision: D14233314

fbshipit-source-id: 8e0ed22c32611e8f6e7d4461c3e31870d47a0e95
2019-02-28 15:42:17 -08:00
Arun Kulshreshtha
9c6b914a22 types: move Key and NodeInfo out of revisionstore
Summary:
In order to move the types in `edenapi-types` (containing types shared between Mercurial and Mononoke) to the `types` crate, we need to move a few types from the  `revisionstore` crate into this crate first, because `revisionstore` depends on `types`, which would create a circular dependency since `edenapi-types` uses types from `revisionstore`.

In particular, this diff moves the `Key` and `NodeInfo` types into their own modules in the `types` crate.

Reviewed By: quark-zju

Differential Revision: D14114166

fbshipit-source-id: 8f9e78d610425faec9dc89ecc9e450651d24177a
2019-02-15 22:51:04 -08:00
Arun Kulshreshtha
575e570a28 bindings: move pyedenapi into bindings crate
Summary: Move the edenapi Python bindings into the common `bindings` crate.

Reviewed By: quark-zju

Differential Revision: D13963179

fbshipit-source-id: 76dead82af992615a9e452ee6fbb9f66639c822c
2019-02-13 16:07:00 -08:00
Arun Kulshreshtha
127ca1a990 edenapi: use rustls instead of openssl
Summary:
Switch from using OpenSSL (via `native-tls`) to [Rustls](https://github.com/ctz/rustls), a pure-Rust TLS implementation based on the `ring` crypto crate.

Unlike `native-tls`, Rustls supports ALPN, which means it can be used along with Hyper to perform HTTP/2 requests over TLS. (OpenSSL also supports ALPN, but older versions of Windows' `schannel` library do not, and as such `native-tls` doesn't support ALPN either regardless of platform.)

Rustls also builds on Windows without any special configuration, sidestepping the issues we've been having with OpenSSL in the Windows build.

Reviewed By: quark-zju

Differential Revision: D14070084

fbshipit-source-id: 25268c58a88177f4708370696d326b4c0bdc89a0
2019-02-13 16:07:00 -08:00
Arun Kulshreshtha
1d00a343e4 bindings: add init module
Summary:
Add a new `init` module to the `bindings` crate. This is intended as a place to put global Rust initialization code for Mercurial's Rust extensions. Ordinarily, such code would go at the start of `main()`, but since `hg` doesn't have a Rust main, putting initialization here at least guarantees that it will happen before any of the Rust extension code runs.

Right now, the only thing initialized in the new module is `env_logger`.

Reviewed By: quark-zju

Differential Revision: D14072560

fbshipit-source-id: 0f2770d0a3a6e9c6b7fe68eb62007cc091adad59
2019-02-13 16:07:00 -08:00
Arun Kulshreshtha
8aaab173c0 bindings: convert to Rust 2018
Summary: Convert crate to Rust 2018.

Reviewed By: quark-zju

Differential Revision: D13963023

fbshipit-source-id: f81a1ecd505d3bdc70c41f5272f861fdd737a8fc
2019-02-05 21:22:48 -08:00
Arun Kulshreshtha
5f6998719e edenapi: add static builder() method to get a client Builder
Summary: In Rust it is typically more idiomatic to have a static method on a struct to produce a builder, since this means the builder doesn't need to be explicitly imported to construct a new instance of the struct.

Reviewed By: DurhamG

Differential Revision: D13925323

fbshipit-source-id: c06d5d42ba941dbbb2c619f9470e79fa23f35f68
2019-02-05 21:22:48 -08:00
Arun Kulshreshtha
70aff50986 edenapi: rename mononokeapi to edenapi
Summary: Rename Mononoke API to Eden API, per war room discussion.

Reviewed By: quark-zju

Differential Revision: D13908195

fbshipit-source-id: 94a2fe93f8a89d0c5e9b6a24939cc4760cfaade0
2019-02-05 21:22:48 -08:00
Arun Kulshreshtha
6eab7a5222 mononokeapi: use auth section for TLS configuration
Summary:
Use the hgrc auth section to configure HTTP data fetching. For example:

```
[mononoke-api]
enabled = true
url = https://mononoke-api.internal.tfbnw.net

[auth]
mononoke-api.prefix = mononoke-api.internal.tfbnw.net
mononoke-api.schemes = https
mononoke-api.key = /var/facebook/credentials/$USER/x509/$USER.pem
mononoke-api.cert = /var/facebook/credentials/$USER/x509/$USER.pem
```

Reviewed By: DurhamG

Differential Revision: D13907185

fbshipit-source-id: 3b607f25912a85438b56c4ea7ab9cfed91ab7e85
2019-02-05 21:22:48 -08:00
Mark Thomas
ae0a81f2c2 rust: move bindings to a single python extension
Summary:
Move all Rust bindings to a single python extension, `bindings`.  This should
improve compilation time and make things simpler.

Reviewed By: quark-zju

Differential Revision: D13923866

fbshipit-source-id: 560592b5a6c0c4f1b836c755ef123666a1059164
2019-02-01 17:53:22 -08:00
Arun Kulshreshtha
50c293b94e pymononokeapi: add Python API for batch file downloads
Summary: Expose `MononokeClient`'s multi-get functionality through the Python bindings using a builder-style API.

Differential Revision: D13893748

fbshipit-source-id: 312e7449eb8300432b1f1de28e0c6fc3e47d1d53
2019-02-01 01:41:31 -08:00
Arun Kulshreshtha
f02ebcffb7 mononokeapi: support fetching multiple files concurrently
Summary: Give MononokeClient the ability to fetch multiple files concurrently. Right now this functionality is not exposed via the Python bindings, so as far as the Mercurial Python code is concerned, nothing has changed. The multi-get functionality will be used later in the stack.

Differential Revision: D13893575

fbshipit-source-id: c9e514fbeb41bbb37f52f6df3920eb01a66df293
2019-02-01 01:41:31 -08:00
Arun Kulshreshtha
1b74af2ace mononokeapi: add ability to fetch a file and write it to a datapack
Summary: Add a new `get_file()` method to `MononokeClient` that fetches Mercurial file content from the API server and writes it to a datapack in the cache. This functionality is exposed via the new `hg debuggetfile` debug command, which takes a filenode and file path and fetches the corresponding file.

Differential Revision: D13889829

fbshipit-source-id: 2b68bf114ee72d641de7a1043cca1975e34cf4e6
2019-02-01 01:41:31 -08:00
Jun Wu
c12e300bb8 codemod: move Python packages to edenscm
Summary:
Move top-level Python packages `mercurial`, `hgext` and `hgdemandimport` to
a new top-level package `edenscm`. This allows the Python packages provided by
the upstream Mercurial to be installed side-by-side.

To maintain compatibility, `edenscm/` gets added to `sys.path` in
`mercurial/__init__.py`.

Reviewed By: phillco, ikostia

Differential Revision: D13853115

fbshipit-source-id: b296b0673dc54c61ef6a591ebc687057ff53b22e
2019-01-28 18:35:41 -08:00