Commit Graph

18 Commits

Author SHA1 Message Date
Alex Hornby
bc85aade21 rust: update to zstd to 0.7.0+zstd.1.4.9
Summary:
Update the zstd crates.

This also patches async-compression crate to point at my fork until upstream PR https://github.com/Nemo157/async-compression/pull/117 to update to zstd 1.4.9 can land.

Reviewed By: jsgf, dtolnay

Differential Revision: D27942174

fbshipit-source-id: 26e604d71417e6910a02ec27142c3a16ea516c2b
2021-04-22 14:34:06 -07:00
Alex Hornby
45f521ddde mononoke: enable default patch.crates-io for internal Cargo.tomls
Reviewed By: quark-zju

Differential Revision: D27915811

fbshipit-source-id: 3f830def66c1c5f0569925c42cc8335ee585e0e7
2021-04-22 10:59:42 -07:00
Stefan Filip
5f78ccb284 edenapi: update hash-to-location to discard unknown hashes
Summary:
The hashes that are passed in as parameters to the hash-to-location function
may not be hashes that actually exist. This change updates the code so that
we don't return an error when an unknown hash is passed in. The unknown
hash will be skipped in the list of results.

Reviewed By: quark-zju

Differential Revision: D27526758

fbshipit-source-id: 8bf9b7a134a6a8a4f78fa0df276f847d922472f5
2021-04-09 17:10:57 -07:00
Stefan Filip
243f858524 mononoke_api: update changeset_ids_to_locations to take multiple master heads
Summary:
This scenario appears when master moves backwards. Since the master group in
segmented changelog is append-only. Non-fast-forward master move will cause
multiple heads in the master group.

Since Segmented Changelog was updated to handle multiple master heads, we can
propagate the full list that we get from the client.

This diff makes the assumption that Mononoke will know to convert all client "master head"
hashes from HgChangesetId (Sha1) form to ChangesetId (Blake2). If any of the master
heads cannot be converted then it means the server might not be reliably answer the
client's question (in "ancestors(master_heads)", translate "this hash" to a path, or tell me
confidently that the "hash" is outside "ancestors(master_heads)"). That's an error case.

Reviewed By: quark-zju

Differential Revision: D27521779

fbshipit-source-id: 219e08a66aac17ac06d2cf02676a43c7f37e8e26
2021-04-09 17:10:57 -07:00
Mark Juggurnauth-Thomas
9e863b0224 test_repo_factory: use test factory for mononoke_api tests
Summary: Use the test factory for existing mononoke_api tests.

Reviewed By: StanislavGlebik

Differential Revision: D27169444

fbshipit-source-id: d940bfa494dfe89fdb891d5248b73ba764f74faf
2021-03-25 07:34:49 -07:00
Carolyn Busch
bd89a4c855 edenapi_server: add bookmark endpoint
Summary: Add the EdenAPI endpoint for resolving bookmarks. This is a first pass that just takes a bookmark name as a path variable, to make sure that this is on the right track. We'll want to add a proper request type that includes a list of bookmarks and a response type that can indicate that no bookmark was found. Then the hg bookmark command will also need support for prefix listing capabilities.

Reviewed By: kulshrax

Differential Revision: D26920845

fbshipit-source-id: 067db6a636a75531ee5953392b734c038a58efb6
2021-03-12 12:07:35 -08:00
Andrey Chursin
0be8e8ce29 vfs: introduce AsyncVfs
Summary:
AsyncVfs provides async vfs interface.
It will be used in the native checkout instead of current use case that spawns blocking tokio tasks for VFS action

Reviewed By: quark-zju

Differential Revision: D26801250

fbshipit-source-id: bb26c4fc8acac82f4b55bb3f2f3964a6d0b64014
2021-03-05 21:47:51 -08:00
Thomas Orozco
2a803fc10d third-party/rust: update futures
Summary:
Those newer versions of Futures have compatibility improvements with Tokio,
notably:

- https://github.com/rust-lang/futures-rs/pull/2333
- https://github.com/rust-lang/futures-rs/pull/2358

Reviewed By: farnz

Differential Revision: D26778794

fbshipit-source-id: 5a9dc002083e5edfa5c614d8d2242e586a93fcf6
2021-03-04 06:42:55 -08:00
Thomas Orozco
ef7045e818 common/rust: use fbinit-tokio
Summary:
This diffs add a layer of indirection between fbinit and tokio, thus allowing
us to use fbinit with tokio 0.2 or tokio 1.x.

The way this works is that you specify the Tokio you want by adding it as an
extra dependency alongside `fbinit` in your `TARGETS` (before this, you had to
always include `tokio-02`).

If you use `fbinit-tokio`, then `#[fbinit::main]` and `#[fbinit::test]` get you
a Tokio 1.x runtime, whereas if you use `fbinit-tokio-02`, you get a Tokio 0.2
runtime.

This diff is big, because it needs to change all the TARGETS that reference
this in the same diff that introduces the mechanism. I also didn't produce it
by hand.

Instead, I scripted the transformation using this script: P242773846

I then ran it using:

```
{ hg grep -l "fbinit::test"; hg grep -l "fbinit::main"  } | \
  sort | \
  uniq | \
  xargs ~/codemod/codemod.py \
&&  yes | arc lint \
&& common/rust/cargo_from_buck/bin/autocargo
```

Finally, I grabbed the files returned by `hg grep`, then fed them to:

```
arc lint-rust --paths-from ~/files2 --apply-patches --take RUSTFIXDEPS
```

(I had to modify the file list a bit: notably I removed stuff from scripts/ because
some of that causes Buck to crash when running lint-rust, and I also had to add
fbcode/ as a prefix everywhere).

Reviewed By: mitrandir77

Differential Revision: D26754757

fbshipit-source-id: 326b1c4efc9a57ea89db9b1d390677bcd2ab985e
2021-03-03 04:09:15 -08:00
Lukas Piatkowski
f317302b0f autocargo v1: reformating of oss-dependencies, workspace and patch sections and thrift files to match v2
Summary:
For dependencies V2 puts "version" as the first attribute of dependency or just after "package" if present.
Workspace section is after patch section in V2 and since V2 autoformats patch section then the third-party/rust/Cargo.toml manual entries had to be formatted manually since V1 takes it as it is.
The thrift files are to have "generated by autocargo" and not only "generated" on their first line. This diff also removes some previously generated thrift files that have been incorrectly left when the corresponding Cargo.toml was removed.

Reviewed By: ikostia

Differential Revision: D26618363

fbshipit-source-id: c45d296074f5b0319bba975f3cb0240119729c92
2021-02-25 15:10:56 -08:00
Thomas Orozco
097e4ad00c mononoke: remove tokio-compat (i.e. use tokio 0.2 exclusively)
Summary:
The earlier diffs in this stack have removed all our dependencies on the Tokio
0.1 runtime environment (so, basically, `tokio-executor` and `tokio-timer`), so
we don't need this anymore.

We do still have some deps on `tokio-io`, but this is just traits + helpers,
so this doesn't actually prevent us from removing the 0.1 runtime!

Note that we still have a few transitive dependencies on Tokio 0.1:

- async-unit uses tokio-compat
- hg depends on tokio-compat too, and we depend on it in tests

This isn't the end of the world though, we can live with that :)

Reviewed By: ahornby

Differential Revision: D26544410

fbshipit-source-id: 24789be2402c3f48220dcaad110e8246ef02ecd8
2021-02-22 09:22:42 -08:00
Lukas Piatkowski
cd0b6d50e2 autocargo v1: changes to match autocargo v2 generation results.
Summary:
The changes (and fixes) needed were:
- Ignore rules that are not rust_library or thrift_library (previously only ignore rust_bindgen_library, so that binary and test dependencies were incorrectly added to Cargo.toml)
- Thrift package name to match escaping logic of `tools/build_defs/fbcode_macros/build_defs/lib/thrift/rust.bzl`
- Rearrange some attributes, like features, authors, edition etc.
- Authors to use " instead of '
- Features to be sorted
- Sort all dependencies as one instead of grouping third party and fbcode dependencies together
- Manually format certain entries from third-party/rust/Cargo.toml, since V2 formats third party dependency entries and V1 just takes them as is.

Reviewed By: zertosh

Differential Revision: D26544150

fbshipit-source-id: 19d98985bd6c3ac901ad40cff38ee1ced547e8eb
2021-02-19 11:03:55 -08:00
Lukas Piatkowski
87ddbe2f74 autocargo v1: update autocargo field format to allow transition to autocargo v2
Summary:
Autocargo V2 will use a more structured format for autocargo field
with the help of `cargo_toml` crate it will be easy to deserialize and handle
it.

Also the "include" field is apparently obsolete as it is used for cargo-publish (see https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields). From what I know this might be often wrong, especially if someone tries to publish a package from fbcode, then the private facebook folders might be shipped. Lets just not set it and in the new system one will be able to set it explicitly via autocargo parameter on a rule.

Reviewed By: ahornby

Differential Revision: D26339606

fbshipit-source-id: 510a01a4dd80b3efe58a14553b752009d516d651
2021-02-12 23:28:25 -08:00
Stefan Filip
93c1231c55 segmented_changelog: update hash_to_location to gracefully handle unknown hashes
Summary:
One of the primary use cases for hash_to_location is translating user provided
hashes. It is then perfectly valid for the hashes that are provided to not
exist.  Where we would previously return an error for the full request if a
hash was invalid, we now omit the hash from the response.

Reviewed By: quark-zju

Differential Revision: D26389472

fbshipit-source-id: c59529d43f44bed7cdb2af0e9babc96160e0c4a7
2021-02-11 12:17:35 -08:00
Stefan Filip
34ff38fe8c mononoke_api: add RepoContext::many_changeset_ids_to_locations
Summary:
Exposes Segmented Changelog changeset to location translation functionality.
This is going to be used by EdenApi endpoints. Those endpoints will be used
by "lazy" Segmented Changelogs on client repositories.

Reviewed By: quark-zju

Differential Revision: D26289620

fbshipit-source-id: 6fb78abab465efd34e2768c8648f438647e9b91c
2021-02-10 10:19:04 -08:00
Stefan Filip
78bc732d5e segmented_changelog: use dag_types::Location for location
Summary:
We had individual fields for location components. This change will make it
easier for people to read through the code. This completes the integration
of the Location struct.

Reviewed By: quark-zju

Differential Revision: D26162272

fbshipit-source-id: 76259578430bac88317afb1935f63e06b6e8284e
2021-02-09 11:31:31 -08:00
Thomas Orozco
497544a63e mononoke/edenapi_service: check & bump load counters for trees & files
Summary:
Like it says in the title. Let's add some basic throttling control here, in
line with what we have in Mononoke Server. The numbers don't quite match up
since fetches in EdenAPI don't include linknodes or history, but this should be
better than nothing and sufficient for now, and makes sense to have with
EdenAPI & Mononoke Server running in the same process.

Reviewed By: HarveyHunt

Differential Revision: D26250746

fbshipit-source-id: 338eda4341a163d0d915f10bf45fc7f40c74fc69
2021-02-05 15:16:09 -08:00
Thomas Orozco
05d54fcb46 mononoke/mononoke_api: split hg parts into mononoke_api_hg
Summary:
I'd like to be able to use mononoke_api from repo_client, but the hg parts
depend on repo_client, so that's a circular dependency.

This splits out the hg parts of Mononoke API so that places that don't want
them don't have to import them. This is similar to what we did with blobrepo.

Reviewed By: StanislavGlebik

Differential Revision: D26099495

fbshipit-source-id: 73a9c7b5dc95feceb35b5eabccf697e9aa0a27de
2021-02-01 07:53:16 -08:00