Commit Graph

35 Commits

Author SHA1 Message Date
Lukas Piatkowski
0d3b4e9301 mononoke args: use cmdlib::args to unify defining and parsing of --myrouter-port flag
Summary: use common funcions to parse --myrouter-port, this is a simple clean up

Reviewed By: StanislavGlebik

Differential Revision: D14084003

fbshipit-source-id: 63d6c8301e977faead62cb1c705bac372d56594e
2019-02-15 13:16:27 -08:00
David Budischek
79cbf5c245 post-commit send scribe message
Summary: New commits should be logged to scribe, these will be used to trigger the update for the hg clone streamfile.

Reviewed By: lukaspiatkowski

Differential Revision: D14022599

fbshipit-source-id: a8a68f12a8dc1e65663d1ccf1a5eafa54ca2daf0
2019-02-13 07:26:33 -08:00
Lukas Piatkowski
515a2909eb monononoke hashes: remove usages of borrows of hashes which are Copy
Summary: The Copy trait means that something is so cheap to copy that you don't even need to explicitly do `.clone()` on it. As it doesn't make much sense to pass &i64 it also doesn't make much sense to pass &<Something that is Copy>, so I have removed all the occurences of passing one of ouf hashes that are Copy.

Reviewed By: fanzeyi

Differential Revision: D13974622

fbshipit-source-id: 89efc1c1e29269cc2e77dcb124964265c344f519
2019-02-06 15:11:35 -08:00
Lukas Piatkowski
44b06df35f hooks: do not pass repo name to hook execution
Summary:
Repo name is used only be verify_integrity hook and even there the name that Mononoke provides is incorrect. Instead of Mononoke's `repo-RepositoryId(1234)` name the hook is interested in Mercurial's `fbsource` name.
HookConfig is a perfect way to pass such an arbitrary parameter so use it.

Reviewed By: StanislavGlebik

Differential Revision: D13964486

fbshipit-source-id: 94090e409d5206828364202ae62a37abc16e4a27
2019-02-06 04:06:19 -08:00
Stanislau Hlebik
0b0a5b93b2 mononoke: extract blobrepo_factory and postcommit from blobrepo
Summary:
`blobrepo_factory` is a crate that knows how to create blobrepo given
a configuration i.e. it creates blobstores, filenodes, changesets etc and
initializes blobrepo with them.

`post_commit` is a small part of blobrepo which can also be extracted from
blobrepo crate.

There are a few upsides with this approach
1) Less dependencies on Blobrepo, meaning we have to rebuild it fewer times
2) BlobRepo compilation is faster

Reviewed By: jsgf

Differential Revision: D13896334

fbshipit-source-id: 1f5701341f01fcefff4e5f9430ddc914b9496064
2019-02-05 01:23:15 -08:00
Stanislau Hlebik
3966192f01 mononoke: split hooks crate
Summary:
The main reason for doing it is to remove dependency on `BlobRepo` from hooks. Most of the  `hooks` crate code needs from `BlobRepo` just a HgBlobChangeset type, which was moved to a separate crate in one of the previous diffs. There is just a small piece of code that depends on blobrepo, and it was moved in the separate crate.

Because of that changing anything in BlobRepo won't trigger rebuilding of most of the hooks crate.

Reviewed By: lukaspiatkowski

Differential Revision: D13878208

fbshipit-source-id: d74336e959282c176258c653d4c408854e1f1849
2019-01-31 00:41:48 -08:00
Stanislau Hlebik
4d48415149 mononoke: split metaconfig crate
Summary:
Currently if a crate depends even on a single type from metaconfig then in
order to compile this trait buck first compiles metaconfig crate with all the
logic of parsing the configs.

This diff split metaconfig into two crates. The first one just holds the types for
"external consumption" by other crates. The second holds the parsing logic.

That makes builds faster

Reviewed By: jsgf, lukaspiatkowski

Differential Revision: D13877592

fbshipit-source-id: f353fb2d1737845bf1fa0de515ff8ef131020063
2019-01-31 00:41:48 -08:00
Stanislau Hlebik
a055f8e409 mononoke: move open_blobrepo to blobrepo crate and remove TestDelay blobrepo type
Summary:
The main reason to do it is to remove dependency from cmdlib to repo_client.
repo_client depends on a lot of other crates like bundle2-resolver, hooks etc.
And it means that in order to compile mononoke_admin we need to compile these
crates too. By moving open_blobrepo into blobrepo crate we are removing
unnecessary dependencies.

Also let's remove unused blobrepo type

Reviewed By: aslpavel

Differential Revision: D13848878

fbshipit-source-id: cd3d04354649cdb5b2947f08762051318725c781
2019-01-30 06:31:10 -08:00
Pavel Aslanov
1eea4fdb09 asynchronous blobrepo constructor
Summary:
- Support for gluster blubstore
- Asynchronous blobrepo constructor

Reviewed By: StanislavGlebik

Differential Revision: D13630033

fbshipit-source-id: 3e7f5a7be4451063dca3df2f5a7f70d30a2def58
2019-01-14 09:31:58 -08:00
Pavel Aslanov
2efacff245 rustfmt effected files
Summary: Format files effected by next commit in a stack

Reviewed By: StanislavGlebik

Differential Revision: D13650639

fbshipit-source-id: d4e37acd2bcd29b291968a529543c202f6944e1a
2019-01-14 09:31:58 -08:00
Stanislau Hlebik
29d1b9e5a0 mononoke: remove Arc<BlobRepo> from hook tailer
Summary: Fix TODO, BlobRepo is cloneable and there is no reason to use Arc<BlobRepo>

Reviewed By: ikostia

Differential Revision: D13607214

fbshipit-source-id: ba280823e8b232d4bf6e62ac2ce8d8cd3ee64c96
2019-01-09 03:52:21 -08:00
Jeremy Fitzhardinge
408e7665d9 mononoke: move RepositoryId into mononoke-types
Summary: There's nothing Mercurial-specific about identifying a repo. This also outright removes some dependencies on mercurial-types.

Reviewed By: StanislavGlebik

Differential Revision: D13512616

fbshipit-source-id: 4496a93a8d4e56cd6ca319dfd8effc71e694ff3e
2018-12-19 10:24:27 -08:00
Harvey Hunt
8bcdf04f1e mononoke: Remove mention of configrepo from arg parsing
Summary:
As we no longer use a config repo, rename the argument
to --config_path.

Reviewed By: rlangst, StanislavGlebik

Differential Revision: D13415205

fbshipit-source-id: 8c7d335373c0ba33131d4476ee72fcd914097385
2018-12-11 05:05:40 -08:00
Stanislau Hlebik
e1dbeb8aac mononoke: remove unused option
Reviewed By: HarveyHunt

Differential Revision: D13339864

fbshipit-source-id: 59b54e10d297966a2e4e7f15138f5826da563f71
2018-12-10 02:29:25 -08:00
Lukas Piatkowski
5d9a151d85 mononoke: pass CoreContext down to blobstore
Reviewed By: jsgf

Differential Revision: D13324220

fbshipit-source-id: af7a2a650338ea66e504ea0acae1d103af10f8e3
2018-12-04 11:40:15 -08:00
Stanislau Hlebik
f07155b402 mononoke: remove mononoke configs, use toml files
Summary:
Config repo proved to be tricky to understand and hard to use. Let's just use
toml files.

Reviewed By: farnz

Differential Revision: D13179926

fbshipit-source-id: 3a44ee08c37284cc4c189c74b5c369ce82651cc6
2018-12-04 03:52:26 -08:00
Lukas Piatkowski
14636545aa mononoke: pass CoreContext down to changesets
Reviewed By: StanislavGlebik

Differential Revision: D13277448

fbshipit-source-id: 6e9a8dac77af8ab991005d14f654e315c234fe44
2018-11-30 10:14:22 -08:00
Lukas Piatkowski
08db0a35eb mononoke: pass CoreContext down to bonsai-hg-mapping
Reviewed By: aslpavel

Differential Revision: D13277450

fbshipit-source-id: 97cfbd917b321727bb4d960c91a784787660eb5b
2018-11-30 10:14:22 -08:00
Tim Fox
d64d9121c5 Allow hook manager cache params to be configured
Summary: ${title}

Reviewed By: StanislavGlebik

Differential Revision: D13082486

fbshipit-source-id: fbdffdbfca218199dc2ac61abfcb8887b299db6d
2018-11-22 03:30:17 -08:00
Stanislau Hlebik
eff25a9f20 mononoke: allow to set limit on the number of processed commits
Summary: Let's allow to set the number of commits processes by hook tailer.

Reviewed By: lukaspiatkowski

Differential Revision: D10361239

fbshipit-source-id: ced118d5dfca3c8aea65cb8a21f5b487f47628cd
2018-10-22 05:22:20 -07:00
Stanislau Hlebik
fb425906e0 mononoke: run hooks in parallel
Summary:
Previously buffered() wasn't particularly useful because it buffered only
mapping from ChangesetId to HgChangesetId. The actual running of hooks was done in
`.and_then()` and that means that each future in the stream should finish
before the next one starts.

Let's put running of hooks inside a buffer, that helps with perf a lot.

Reviewed By: jsgf

Differential Revision: D10359546

fbshipit-source-id: 48b8b200d7397eef8622c32cad9cec889b96f9d0
2018-10-17 08:06:06 -07:00
Stanislau Hlebik
bfe352bae4 mononoke: change logging in hook_tailer
Summary:
Let's change level of logging for each hook/commit to debug, since it was
spammy. Instead let's print a total statistics about how many hooks were
accepted/rejected

Reviewed By: jsgf

Differential Revision: D10358786

fbshipit-source-id: 2e451d482ed5549e41975f9e3b57b05d90069788
2018-10-17 08:06:06 -07:00
Stanislau Hlebik
eaf156206c mononoke: make continuous running optional
Summary:
Let's add an option `--continuous`. If it's not specified then hook tailer runs
only once. That's useful for testing the new hooks.

Reviewed By: jsgf

Differential Revision: D10358785

fbshipit-source-id: b62d01b4bf3233c3f411fc298fefb79da473d7f1
2018-10-17 08:06:05 -07:00
Lukas Piatkowski
cad69fedd0 filenodes: use sqlfilenodes instead of dieselfilenodes; pass myrouter_port around
Reviewed By: farnz

Differential Revision: D10338868

fbshipit-source-id: 60734d9635df442691cad3637aebd5bc838e03ad
2018-10-11 10:52:05 -07:00
Stanislau Hlebik
3c077a3875 mononoke: fix hook tailer
Summary: It got a bit rusty, this diff adds missing stuff

Reviewed By: farnz

Differential Revision: D10302729

fbshipit-source-id: af598f8c8fdd5c938c07052c03ab0f84fc6d3c20
2018-10-11 05:50:48 -07:00
Stanislau Hlebik
a79d9d4a25 mononoke: add pushvars bypasses
Summary: Pushvars is a one more way to bypass hooks. This diff implements it

Reviewed By: purplefox

Differential Revision: D10257602

fbshipit-source-id: 1bd188239878ff917ded7db995ea2453da9f64c4
2018-10-11 05:50:48 -07:00
Lukas Piatkowski
1d69b1f884 mononoke: add a flag for --myrouter-port, ignore it for now
Summary:
The idea for rollout is to:
- first make sure that Mononoke doesn't crash when a --myrouter-port is provided
- then tupperware configs will be modified to include myrouter as a collocated proces on every host and the port of that myrouter instance will be provided via command line
- lastly land the change that actually talks to myrouter

Reviewed By: StanislavGlebik

Differential Revision: D10258251

fbshipit-source-id: ea9d461b401d41ef624304084014c2227968d33f
2018-10-09 10:21:04 -07:00
Tim Fox
debd4a5f95 Integrate hooks into mononoke write path
Summary: Integrate hook manager into the Mononoke pushrebase path

Reviewed By: lukaspiatkowski

Differential Revision: D9896005

fbshipit-source-id: ffd79b539288d95b134af97b776b7fcc3afa0ce7
2018-10-02 07:07:16 -07:00
Stanislau Hlebik
8297747e3b mononoke: make AncestorsNodeStream use ChangesetFetcher
Summary:
Main reason is to make startup faster because ChangesetFetcher can uses bulk
caches in the same way as getbundle does.

Reviewed By: farnz

Differential Revision: D10032435

fbshipit-source-id: 717114339edf31865b498893d75695968447bb43
2018-09-26 04:51:45 -07:00
Stanislau Hlebik
7cec9d7878 mononoke: make AncestorNodeStream use ChangesetId
Summary:
Revsets should use ChangesetId instead of NodeHash. This diff cleans up
ancestors revset

Reviewed By: farnz

Differential Revision: D10032436

fbshipit-source-id: 2c7d170738826154e3b606e9e29a739a34b1840e
2018-09-26 04:51:45 -07:00
Tim Fox
35c734de1d Support access to file content in changeset hooks
Summary:
This diff enables access to file content via contains_string() and len() (same as in file hooks) form inside changeset hooks.
This is necessary as some changeset hooks need access to file content and length, e.g. to compute total changeset size.

Reviewed By: StanislavGlebik

Differential Revision: D9788596

fbshipit-source-id: da7bafe6f6fa17a1f25b42550d0bb1a5d871579e
2018-09-25 04:13:50 -07:00
Pavel Aslanov
57d5ddcaf8 added pushrebase configuration options
Summary:
- added `PushrebaseParams` to `RepoConfig`
- configurable recursion_depth and rewritedates

Reviewed By: StanislavGlebik

Differential Revision: D9578661

fbshipit-source-id: df26be4f0f54a54ab6a82fc89d6733099469ce98
2018-08-31 08:55:19 -07:00
Rain ⁣
7f039b5c63 repo_client: Arc<BlobRepo> -> BlobRepo
Summary: Remove `Arc<BlobRepo>` from more places since `BlobRepo` will do its own internal `Arc`ing.

Reviewed By: StanislavGlebik

Differential Revision: D9317987

fbshipit-source-id: 899e8b2ede278e62a83e64c144eb18c8cc7e57c6
2018-08-15 23:36:18 -07:00
Jeremy Fitzhardinge
1968ad7984 mononoke: set panichandler in hook_tailer
Summary:
Make panics more obvious by printing a backtrace and aborting the
process.

Reviewed By: lukaspiatkowski

Differential Revision: D9282766

fbshipit-source-id: 9e22c1ced5a651dda3b58fa3dd84465699235d2c
2018-08-13 14:37:02 -07:00
Tim Fox
a82b90c6f9 Implement hook tailer
Summary:
This diff implements a service which tails a configurable bookmark and runs hooks against it.
It uses the standard Mononoke config from the meta config repo.

Reviewed By: StanislavGlebik

Differential Revision: D8898637

fbshipit-source-id: f710fe4c9bda1b78bd17eb6cd6abf2abda4fdb8e
2018-08-08 05:36:13 -07:00