Commit Graph

63252 Commits

Author SHA1 Message Date
Alex Hornby
5f01efbe75 mononoke: return shard count from MetadataSqlFactory::tier_name_shardable and rename it
Summary: Expose a way to get the shard count for a table.  Used later by walker to limit number of steps that try to access the same SQL shard.

Differential Revision: D26497839

fbshipit-source-id: 45ad09527b82a36a76068aace5fff5ef504b7a12
2021-02-19 07:55:43 -08:00
Thomas Orozco
b0021495a2 mononoke/mercurial: remove Tokio 0.1 dependency
Summary:
This just moves the cheese to use Tokio 0.2 spawn. I have some extra diffs
to actually tackle the code itself here, but I'm planning to do this later.

Reviewed By: StanislavGlebik

Differential Revision: D26511981

fbshipit-source-id: 249d0e667f9930272c9362fc787c977d8d50341e
2021-02-19 07:00:55 -08:00
Thomas Orozco
767c961fa4 mononoke/mononoke_hg_sync_job: update to Tokio 0.2 Hyper
Summary:
Like it says in the title, this updates us to a newer version of Hyper to avoid
being on one that depends on Tokio 0.1.

Reviewed By: StanislavGlebik

Differential Revision: D26511979

fbshipit-source-id: 325d24f9b4e17fc2e801a2ba79863c0e656870d4
2021-02-19 07:00:55 -08:00
Thomas Orozco
0490fd9622 mononoke_hg_sync_job: update hg peer to tokio 0.2
Summary:
This code was a bit hard to convert because just using the 0.2 variants really
doesn't work very well. So, I went ahead and actually refactored it. Here's
what I changed:

- Rather than use a `!Sync` bound to ensure we don't have concurrent access to
  the `HgPeer`, I updated this to actually use a `&mut` reference in an `async
  fn`. Note that the `!Sync` bound doesn't really do anything here because
  it prevented you from instantiating a future concurrently in 2 threads but
  nothing prevents you from creating 2 futures and awaiting them concurrently.
  The `&mut` reference does (and means we have to wrap this in a Tokio mutex).
- I moved the management of `invalidate()` to the `HgPeer` and not
  `AsyncProcess`, given it's always the `HgPeer` that makes the decision to
  invalidate.

Reviewed By: StanislavGlebik

Differential Revision: D26511980

fbshipit-source-id: 6deb5be76effbb65cef29c789b9b3e4429326c5f
2021-02-19 07:00:55 -08:00
Thomas Orozco
f18d98939b mononoke/mononoke_hg_sync_job: rename 0.1 futures_ext
Summary:
I want to use both future_ext libraries later in this stack, and this will
make it easier.

Reviewed By: StanislavGlebik

Differential Revision: D26510582

fbshipit-source-id: c90fca327697e0e966e8d6ac262115ee69c99112
2021-02-19 07:00:54 -08:00
Thomas Orozco
72ed8767e0 mononoke/unbundle: remove tokio 0.1
Summary: Here again, easy, it's not used.

Reviewed By: StanislavGlebik

Differential Revision: D26485714

fbshipit-source-id: 1f96e05934c4b649d862ce992ca90b031ea241a7
2021-02-19 07:00:54 -08:00
Thomas Orozco
3b636d4780 mononoke/revset: remove tokio 0.1 dependency
Summary:
This depends on Tokio for a reexport of futures_old::task (this can't be from
Tokio, and has to live in `future`s, or it wouldn't be possible for futures to
provide `.compat()` on a 0.3 future).

Reviewed By: StanislavGlebik

Differential Revision: D26485715

fbshipit-source-id: 6f334132eb7c7aaa647f8e5052e91c3a38f6d1d8
2021-02-19 07:00:54 -08:00
Thomas Orozco
012ecf21db mononoke/repo_listener: remove tokio 0.1
Summary: Easy: it's not used.

Reviewed By: StanislavGlebik

Differential Revision: D26485713

fbshipit-source-id: 39de39f73ebf5b53f5eaecfc8fa5cf5aeb1d88e8
2021-02-19 07:00:53 -08:00
Thomas Orozco
3d4e09a4d5 eden/mononoke/commit_rewriting/backsyncer: remove tokio 0.1 spawns
Summary:
This is using `spawn_future` (aka tokio 0.1 spawn) so it needs changing.
Amusigly, this uses it to spawn a 0.2 future and then it has to pull 0.1
futures to poll it.

Well, anyhow, this fixes this, this removes all the manual instantiation
of a compat runtime (because this never needed one), and it also removes 0.1
futures (because we never needed them).

Reviewed By: StanislavGlebik

Differential Revision: D26485584

fbshipit-source-id: f49c85d2a1dc406e2ad9e7998d9270fe2e6c9f4b
2021-02-19 07:00:53 -08:00
Thomas Orozco
69d7f598d7 mononoke/blobimport_lib: remove spawn_future
Summary:
Like it says in the title. This is a Tokio 0.1 function, and we want the 0.2
version instead.

Reviewed By: StanislavGlebik

Differential Revision: D26485579

fbshipit-source-id: 95abb28520a61b6dc44a4223a3c6585675a33f3e
2021-02-19 07:00:53 -08:00
Thomas Orozco
d2115eed3c mononoke/blobimport_lib: remove tokio 0.1 dependency
Summary: Like it says in the title.

Reviewed By: StanislavGlebik

Differential Revision: D26485581

fbshipit-source-id: 53bc2aea96ef7c073595ba85409f7f6c9b2da516
2021-02-19 07:00:53 -08:00
Thomas Orozco
799e7809a6 mononoke/blobimport_lib: rename futures_ext alias
Summary: I need to use both futures_ext later in this stack. This makes it easier.

Reviewed By: StanislavGlebik

Differential Revision: D26485580

fbshipit-source-id: 589713c7280bf74c9e2b3e018782bf0b7eb821d7
2021-02-19 07:00:52 -08:00
Thomas Orozco
3073987faf mononoke/repo_client: remove 0.1 tokio timeout from getcommitdata
Summary: This lets us ensure we only depend on Tokio 0.2 for timers.

Reviewed By: StanislavGlebik

Differential Revision: D26485590

fbshipit-source-id: e39dbc21dc51070113b3c9df497c2e0bbaa12450
2021-02-19 07:00:52 -08:00
Thomas Orozco
ae83446c36 mononoke/repo_client: remove 0.1 tokio timeout from streaming clone
Summary:
Like it says in the title. This one was pretty easy to just convert to 0.3
futures so I did so.

Reviewed By: StanislavGlebik

Differential Revision: D26485577

fbshipit-source-id: 76c751c1004288dda1d7b62866979c9228e0ef34
2021-02-19 07:00:52 -08:00
Thomas Orozco
c8854bf5c3 mononoke/repo_client: remove 0.1 tokio timeout in getbundle / gettreepack
Summary:
Like it says in the title. Porting anything to Futures 0.3 isn't practical
here so I didn't touch it.

Reviewed By: StanislavGlebik

Differential Revision: D26485585

fbshipit-source-id: 291bf63a6f31d502ac14492151f14bae20009094
2021-02-19 07:00:51 -08:00
Thomas Orozco
bfe01641ad mononoke/repo_client: remove 0.1 tokio timeout from getpack
Summary: Like it says in the title.

Reviewed By: StanislavGlebik

Differential Revision: D26485573

fbshipit-source-id: f0604760ff73352c7e3601103a84619186cac0d7
2021-02-19 07:00:51 -08:00
Thomas Orozco
d047316a8e mononoke/repo_client: rename 0.1 stream:: to stream_old::
Summary: This will make the next diff easier to read.

Reviewed By: StanislavGlebik

Differential Revision: D26485586

fbshipit-source-id: c52d5355c8d9ed742b4de0b1faab460ef6664c69
2021-02-19 07:00:51 -08:00
Thomas Orozco
6583143fec mononoke/repo_client: remove direct usage of Tokio 0.1
Summary:
Like it says in the title. This removes places where we use Tokio 0.1 directly
in repo client. We use it for timeouts, so this updates us to Tokio 0.2
timeouts.

Where possible, I've made a few improvements to the code as well:

- I removed timeouts on `future::ok()` because a future that is immediately
  ready isn't going to time out.
- I updated some code to async / await where it made sense to do so to avoid
  round-tripping through futures 0.1 and 0.2 several times.

One thing that changes here is that we'll show Tokio's error on timeouts (which
says timeout has elapsed) instead of ours, which used to say just "timeout". I
think this doesn't make a big difference.

Reviewed By: StanislavGlebik

Differential Revision: D26485575

fbshipit-source-id: 8158f709bcc52d123a95df541aaeb1ec0fc9c281
2021-02-19 07:00:50 -08:00
Thomas Orozco
fc48f40f4a mononoke: update futures_ext name in repo_client
Summary: I'd like to use the 0.3 version here so let's get this cleaned up.

Reviewed By: StanislavGlebik

Differential Revision: D26485583

fbshipit-source-id: 1d1ff8e75888e6d874d21195cae7600f171321ac
2021-02-19 07:00:50 -08:00
Thomas Orozco
e9325e986a mononoke: remove tokio-old from blobrepo_utils
Summary:
This just spawns stuff. I took a look at also updating the underlying future
to 0.2, but it wasn't trivial so that's better left off as a future task.

Reviewed By: StanislavGlebik

Differential Revision: D26485588

fbshipit-source-id: 02eb6221c36a4b81bf5dae0232be39ae1dd8201a
2021-02-19 07:00:50 -08:00
Thomas Orozco
f8d1522377 mononoke: remove tokio-01 from blobstore
Summary: Easy: it wasn't used.

Reviewed By: StanislavGlebik

Differential Revision: D26485587

fbshipit-source-id: ab4a1e33eb1c9d6ce193fd5b7a157de45011e947
2021-02-19 07:00:50 -08:00
Thomas Orozco
737e98580e common/rust/shed/futures_ext: split FbFutureExt
Summary:
For streams we have `FbStreamExt` and `FbTryStreamExt`. Let's be a little
consistent and do the same with the future extension trait.

Reviewed By: StanislavGlebik

Differential Revision: D26485589

fbshipit-source-id: 5ebbda11d02e16709958a99a806aa70a8354672e
2021-02-19 07:00:49 -08:00
svcscm
03570dc83e Updating submodules
Summary:
GitHub commits:

f1a22380a6
d35005908b

Reviewed By: bigfootjon

fbshipit-source-id: 68a89a97a738056b33a827ea5c5ab3041559ca86
2021-02-19 06:45:18 -08:00
svcscm
fd11be440e Updating submodules
Summary:
GitHub commits:

d5e7cdca1e
6029ab003f
4d9f2e6fc2
88b952705f
79ac32dcae
3942f3df2e
8665e49abf

Reviewed By: bigfootjon

fbshipit-source-id: 5f57ff80ed3dd64d01e6bc5462d4d4a28afb57c0
2021-02-19 06:45:18 -08:00
svcscm
7db0f58580 Updating submodules
Summary:
GitHub commits:

bc880a08d3
8599380a59
c9878baa87
f38de5367c

Reviewed By: bigfootjon

fbshipit-source-id: e2e5e49dfe29176122457d89201973922e4c3697
2021-02-18 20:06:33 -08:00
Aida Getoeva
bc392c1dfa mononoke/hooks: extend file content manager to search for last updates
Summary:
OWNERS hook sometimes needs to know the most recent changes made to the OWNERS files. So the user, who was last to change the OWNERS file, can change it again.

This diffs extends `FileContentManager` trait to have a new method `recent_changes`. It derives unode manifest for the given bookmark, finds given paths and gets linknodes. For the linknodes it derives `ChangesetInfo` which is a lightweight version of the bonsai changesets, they contain all the metadata, but don't have a set file changes.

We use unodes here instead of hg manifests, because hg manifests don't have correct linknodes.

Reviewed By: StanislavGlebik

Differential Revision: D26468603

fbshipit-source-id: 78dd0a4e51bee6228de1296eafe7ffa393df2fbe
2021-02-18 18:46:26 -08:00
svcscm
d939465e4d Updating submodules
Summary:
GitHub commits:

bfda1efa54
1d97d5b609
ad25b1afb9
f29e003ee4

Reviewed By: bigfootjon

fbshipit-source-id: 88b8e8a09b7d0c501e0ef566a5816c69ab38032a
2021-02-18 18:09:14 -08:00
Johan Schuijt-Li
22c1735c5e mononokepeer: don't concat strings when help message not set
Summary:
Fix for error:
  can only concatenate str (not "NoneType") to str

Reviewed By: quark-zju

Differential Revision: D26523879

fbshipit-source-id: 3d15d72be3444df25fb705ce444d6c3621547551
2021-02-18 18:00:27 -08:00
svcscm
57ae2e77a7 Updating submodules
Summary:
GitHub commits:

cd365bcd01
337eca866b
59ba104e4a

Reviewed By: bigfootjon

fbshipit-source-id: b2f0815a5d08236b80b2f9c202cbc5cd9b767e27
2021-02-18 11:52:36 -08:00
Alex Hornby
97a7a08637 mononoke: update MetadataSqlFactory methods to async
Summary: makes it a bit easier to work with

Reviewed By: farnz

Differential Revision: D26497840

fbshipit-source-id: bccd248aa9a4caa4e5274f7f9e166ef17196e12f
2021-02-18 08:11:47 -08:00
Alex Hornby
ac4c815e79 mononoke: remove Clone from samplingblob
Summary: Noticed we could relax this constraint and the code could also be cleaned up a little.

Reviewed By: farnz

Differential Revision: D26490859

fbshipit-source-id: 27d3a0e308e593dfa87d5c7d7b47736947f659a8
2021-02-18 08:05:38 -08:00
Alex Hornby
b7a245dad7 mononoke: remove Clone from CacheBlobstore inner
Summary: Noticed we can relax this constraint

Reviewed By: farnz

Differential Revision: D26490862

fbshipit-source-id: 59e9af6e2b8ebd433f9536e8c0ebec3f401e360e
2021-02-18 08:05:38 -08:00
Kostia Balytskyi
ce88c33192 observability: import thrift changes
Summary: Just import D26423954

Reviewed By: farnz

Differential Revision: D26482499

fbshipit-source-id: 4a1b2354109bfdb84ad07a167ec206159751c203
2021-02-18 07:35:40 -08:00
svcscm
bd6924ef48 Updating submodules
Summary:
GitHub commits:

ae75fb8676
8ad97858a4
3a03f68a6b
975e41df81

Reviewed By: bigfootjon

fbshipit-source-id: e6bf3863c4af5a46cc717e34bb33fca2275c40b1
2021-02-18 07:35:39 -08:00
svcscm
c5d085a2c5 Updating submodules
Summary:
GitHub commits:

f54fb56f35
e6c0839ede

Reviewed By: bigfootjon

fbshipit-source-id: 6e1ae9901d762e8cd2f293de8de980551f9acd17
2021-02-18 04:40:04 -08:00
Liubov Dmitrieva
84280e36c3 remove accessed bookmarks feature
Summary:
Remove selectivepullaccessedbookmarks feature because it contains bugs and causes many undesired issues.

This was added to migrate existing repos to selective pull and is not needed anymore.

Main effects are:

* if you enable selectivepull for an existing repo, it won't reduce number of subscribed bookmarks.
* some operations like `hg push` or `hg pull -r` in their underlying implementation update all subscribed bookmarks, not just accessed like before.
This drives changes to the tests. Reminder, the bookmark has been marked as "accessed" if the repo has been ever updated to that bookmark.

All tests fixed accordingly.

Reviewed By: markbt

Differential Revision: D26460435

fbshipit-source-id: f839b9f207bfc478a0336ec807b720d35a0bb12e
2021-02-18 04:02:06 -08:00
Mark Juggurnauth-Thomas
08a4201285 cmdlib: initialize cache pool for bonsai_svnrev_mapping
Summary: Add a cache pool for the new bonsai_svnrev_mapping

Reviewed By: krallin

Differential Revision: D26511043

fbshipit-source-id: feaa1bca525e84ebea7462bcc9a814dcb9e5a478
2021-02-18 03:22:31 -08:00
svcscm
9e526526a8 Updating submodules
Summary:
GitHub commits:

edc8b7ec57
23e8e27146
f62a4ba3bd
ee83817379
cadbb5f3d7
f308846a56
c2022e9705
b78d624695
ca19d6686d
92a49804a9
0f845a6d28
e4863ffc57

Reviewed By: bigfootjon

fbshipit-source-id: 78bc0bf30669ef5f0b82c585a2365fd77dd7cfaf
2021-02-18 03:22:31 -08:00
svcscm
962441dfd8 Updating submodules
Summary:
GitHub commits:

9ed59bee41
50b243f35d
3574a37653
0effaa946a
f9ffe50a8b
6a85aea5b1
75ce606cdc
6ab6ed156e
8b3d0e0411
ad7c0da314
47c6136c57
8a8dec3222
3127d8ff4d

Reviewed By: bigfootjon

fbshipit-source-id: 3f962fef3f883327eef0b0d38ac835236684a4dc
2021-02-18 00:00:17 -08:00
Xavier Deguillard
ce50d2e34c fs: add an NfsDispatcher
Summary:
Similarly to what is done for FUSE and ProjectedFS, the dispatcher is the glue
that sits in between the protocol specific bits and the inodes layer.

For now, this only implements "getattr" but it will be filled overtime as more
RPC can be answered properly.

Reviewed By: kmancini

Differential Revision: D26389795

fbshipit-source-id: 19cf3457feec2ebc100e632cb28c20b11fdde26d
2021-02-17 23:32:38 -08:00
Xavier Deguillard
992d17bd2d nfs: add getattr RPC datastructures
Summary: This merely adds the various datastructures needed to implement GETATTR.

Reviewed By: kmancini

Differential Revision: D26389794

fbshipit-source-id: bda557a21386483449c18aa9e52f4f626b73e69f
2021-02-17 23:32:38 -08:00
Yedidya Feldblum
61ce8b9ba2 getdeps show-build-dir
Summary: A show-build-dir subcommand. Possibly useful for building a project and then running only one test or benchmark binary directly.

Reviewed By: wez

Differential Revision: D26502869

fbshipit-source-id: 0d8f6c42a2bbeb2440503a39c7fc770e3b3e4fff
2021-02-17 20:15:35 -08:00
svcscm
a17fb634fb Updating submodules
Summary:
GitHub commits:

9643afe114
2620a85b1c
cb2d12ebd3
f59c2a6b34
dab4fe5bcd
751173fb69
e3ffd60749
0405490239
887359c3ed

Reviewed By: bigfootjon

fbshipit-source-id: 6be1f33cf04a1f118e71174586c11c3a806e6bb4
2021-02-17 18:07:30 -08:00
Durham Goode
5b5eec156c treemanifest: remove logic that replaces the stores
Summary:
This was added in D26387205 (ca1249c269) to make the treemanifest store usage look
more like the file store usage, but it appears to cause read-after-write
failures. I'm not sure exactly why, but I think it's because something takes a
reference to the old store and writes to it, then something else reads from the
new store. Or vice versa.

Either way, reverting these three lines fixes it.

Reviewed By: singhsrb

Differential Revision: D26497708

fbshipit-source-id: e9afabed8d5f800260e647d5ed1513eed0bc60cd
2021-02-17 14:36:14 -08:00
Mateusz Kwapich
d46aee9954 add support for svn revision numbers source control service
Summary: Let's treat them like any other ids.

Reviewed By: StanislavGlebik

Differential Revision: D26453408

fbshipit-source-id: fb01bf9ca8871a147a41dded7eed6e3360466a44
2021-02-17 12:43:01 -08:00
Mateusz Kwapich
db7a5abbf6 add svn rev numbers to mononoke_api crate
Summary: just some boilerplate to make the use from SCS code clean and easy

Reviewed By: StanislavGlebik

Differential Revision: D26453406

fbshipit-source-id: efc280e0be14c3e0f145a489a9fd77b5c0eefa58
2021-02-17 12:43:01 -08:00
Mateusz Kwapich
3b32b1629f make backfill_git_mapping more generic and supporing svn
Summary: backfill_git_mapping was great for backfilling git mappings, let's use it for svn

Reviewed By: StanislavGlebik

Differential Revision: D26453407

fbshipit-source-id: 3ee61722ac729af5a2fc4187fda3341069bb53f6
2021-02-17 12:43:00 -08:00
Mateusz Kwapich
adba5b3e30 add BonsaiSvnrevMapping to blobrepo
Summary: now that we have the bonsai svn mapping let's thread it through blobrepo

Reviewed By: StanislavGlebik

Differential Revision: D26453409

fbshipit-source-id: 4100a9b16ffb3186735e20a766a1ef486d3c26c5
2021-02-17 12:43:00 -08:00
Mateusz Kwapich
45465ba8d8 make BonsaiSvnrevMapping store the repo_id
Reviewed By: krallin

Differential Revision: D26467968

fbshipit-source-id: 844f2d43fc0541e8aaffdc40ea9f876e3432e3c5
2021-02-17 12:43:00 -08:00
Mateusz Kwapich
ccf4dad352 introduce svn revision mapping
Summary:
Bonsai svnrev mapping will allow us to store the original svn revision number for repos that
were imported from svn at some point in time. This will allow scs_server to
easily respond to queries about commits identified by them.

The code is mostly copypasta from globalrevs

Reviewed By: krallin

Differential Revision: D26424833

fbshipit-source-id: 9825de52c23ab9386de1a9d0e0a9506c0a035a8a
2021-02-17 12:43:00 -08:00