Commit Graph

113 Commits

Author SHA1 Message Date
Thomas Orozco
77ba80ebd8 mononoke: Rename Bookmark to BookmarkName
Summary:
As part of adding support for infinitepush in Mononoke, we'll include additional server-side metadata on Bookmarks (specifically, whether they are publishing and pull-default).

However, we do use the name `Bookmark` right now to just reference a Bookmark name. This patch updates all reference to `Bookmark` to `BookmarkName` in order to free up `Bookmark`.

Reviewed By: StanislavGlebik

Differential Revision: D15364674

fbshipit-source-id: 126142e24e4361c19d1a6e20daa28bc793fb8686
2019-05-21 12:26:02 -07:00
Thomas Orozco
e6ba3a36c3 mononoke: rustfmt hooks/src/lib.rs
Summary: I'm going to be doing some work on this file, but it's not up-to-date with rustfmt. To minimize merge conflicts and simplify diff reviews, I ran that earlier.

Reviewed By: StanislavGlebik

Differential Revision: D15364675

fbshipit-source-id: 66e3d2287ffcaf93bc6ad17f0217517f0ddb9b2f
2019-05-21 12:25:56 -07:00
Jun Wu
150f6f7f6b mononoke: add bookmark config to disable pushrebase date rewrite
Summary:
There was a request about importing a GitHub repo into fbsource. While pushing
it to Mononoke with pushrebase disabled, the sync job broke because it can only
handle pushrebase pushes.

Before this diff, pushrebase has a repo-level config about whether dates need
to be rewritten. We definitely want "master" to have date rewritten turned on,
but not the imported commits. This diff adds logic to turn off date rewriting
for bookmarks by using the `rewrite_dates` config, to address the repo import
requirement.

Reviewed By: StanislavGlebik

Differential Revision: D15291030

fbshipit-source-id: 8dcf8359d7de9ac33f0af6f9ab3bcbac424323e4
2019-05-21 12:25:55 -07:00
Jeremy Fitzhardinge
046abb21ad mononoke: Migrate to new config structures, leaving config files unchanged
Summary:
This migrates the internal structures representing the repo and storage config,
while retaining the existing config file format.

The `RepoType` type has been replaced by `BlobConfig`, an enum containing all
the config information for all the supported blobstores. In addition there's
the `StorageConfig` type which includes `BlobConfig`, and also
`MetadataDBConfig` for the local or remote SQL database for metadata.

Reviewed By: StanislavGlebik

Differential Revision: D15065421

fbshipit-source-id: 47636074fceb6a7e35524f667376a5bb05bd8612
2019-05-21 12:25:43 -07:00
Stanislau Hlebik
b42c5a2b66 mononoke: set bookmark for hooks and use it in verify_integrity
Summary:
At the moment verify integrity script blocks master commits that do not pass
the check. Non-master commits are not blocked even if they don't pass the check
(for example, if they don't have a valid reviewer), but verify_integrity hook
still logs them to scuba.

Mononoke's verify_integrity was enabled only on master, meaning that the hook
won't run on non-master commits at all, and we won't log non-master commits to scuba.
This diff fixes it.

Reviewed By: farnz

Differential Revision: D15146725

fbshipit-source-id: ab432fb2eccae0fbcc10755f5c8447964c490285
2019-05-21 12:25:29 -07:00
Pavel Aslanov
3b6081a6f5 make it possible to enable bookmark cache from configs
Summary: Now it is possible to configure and enable/disable bookmark cache from configs

Reviewed By: StanislavGlebik

Differential Revision: D14952840

fbshipit-source-id: 3080f7ca4639da00d413a949547705ad480772f7
2019-05-21 12:25:22 -07:00
Zeyi (Rice) Fan
6bc1af095d update to Rust 1.34.0
Reviewed By: jsgf

Differential Revision: D14963608

fbshipit-source-id: 550d63cd10c52e8bcbb25137bc29811955da0b20
2019-05-21 12:25:14 -07:00
Pavel Aslanov
7ca65d1da3 restrict bookmarks moves onty to allowed set of users
Summary: restrict bookmarks moves onty to allowed set of users

Reviewed By: farnz

Differential Revision: D14934606

fbshipit-source-id: d149824b4d3d6376dde6e855cac214e5fda89fac
2019-05-21 12:25:11 -07:00
Pavel Aslanov
676dbe9f75 check verify_integrity binary existence on hook loading instead of on runtime
Summary: - check verify_integrity binary existence on hook loading instead of on runtime

Reviewed By: HarveyHunt

Differential Revision: D14874266

fbshipit-source-id: 77ee7b6c8fb56f84fc8621ace3b68901c934ba34
2019-05-21 12:25:07 -07:00
Pavel Aslanov
2f9040197e convert crate to rust-2018
Summary: - convert crate to rust-2018

Reviewed By: StanislavGlebik

Differential Revision: D14873761

fbshipit-source-id: 992cba12179a1fdfbcc05aa9dae24e04af81de2e
2019-05-21 12:25:07 -07:00
Pavel Aslanov
9aae3e5869 new restrict_users hook
Summary:
We need this hook to replicate functionality of [disable-nonff.py](diffusion/OPSFILES/browse/master/chef/cookbooks/other/fb_mercurial_server/files/default/scripts/hooks/bin/disable-nonff.py;84ce14ceebcb6ef47b0c32feb0db742439e0ffb5$46-49)
I is now possible to restrict users from moving specified bookmark

Reviewed By: StanislavGlebik

Differential Revision: D14871808

fbshipit-source-id: e2f7e8b97f789cfaff2d76abc405e8bd1c6abdd8
2019-05-21 12:25:06 -07:00
Harvey Hunt
933f7facaf mononoke: Always print hook rejections from hook_tailer
Summary:
Previously, the hook_tailer would only print hook rejections if
the --debug flag was passed. Modify the tailer so that it always prints
out rejections.

Reviewed By: farnz

Differential Revision: D14799244

fbshipit-source-id: 3b2c5b00b6cfa54f6fa93c58406b1720876fd9d4
2019-05-21 12:25:03 -07:00
Johan Schuijt-Li
04845a1298 hooks: system users can also be reviewers
Reviewed By: aslpavel

Differential Revision: D14643789

fbshipit-source-id: 050fd211d6b99c9e55461d02b6f71b491b49f770
2019-05-21 12:24:53 -07:00
Zeyi (Rice) Fan
cada677bc6 Update to Rust 1.33.0
Summary: Update Rust toolchain to 1.33.0 with fixes to make our code compatible with 1.33.0.

Reviewed By: Imxset21, kulshrax

Differential Revision: D14608312

fbshipit-source-id: 2d9cf7d01692abaed32f9adffa0e5eb51cfacb4f
2019-05-21 12:24:52 -07:00
David Budischek
2a93fe345c Block non fastforward bookmark moves
Summary:
This is a hook in mercurial, in Mononoke it will be part of the implementation. By default all non fastforward pushes are blocked, except when using the NON_FAST_FORWARD pushvar (--non-forward-move is also needed to circumvent client side restrictions). Additionally certain bookmarks (e.g. master) shouldn't be able to be moved in a non fastforward manner at all. This can be done by setting block_non_fast_forward field in config.

Pushrebase can only move the bookmark that is actually being pushrebased so we do not need to check whether it is a fastforward move (it always is)

Reviewed By: StanislavGlebik

Differential Revision: D14405696

fbshipit-source-id: 782b49c26a753918418e02c06dcfab76e3394dc1
2019-03-18 04:12:09 -07:00
David Budischek
16bfb3eb45 Remove unnecessary Option in RepoConfig
Summary: There is no need to have an Option<Vec<XYZ>> as None can simply be represented by an empty vector. This makes these fields easier to use.

Reviewed By: StanislavGlebik

Differential Revision: D14405687

fbshipit-source-id: e4c5ba12a1e3c6a18130026af6814d54952da4d2
2019-03-18 04:12:09 -07:00
David Budischek
7b92ba7c25 Remove logging to scribe from blobrepo
Summary: See D14279065, this diff is simply to clean up the deprecated code

Reviewed By: StanislavGlebik

Differential Revision: D14279210

fbshipit-source-id: 10801fb04ad533a80bb7a2f9dcdf3ee5906aa68d
2019-03-12 04:50:45 -07:00
Lukas Piatkowski
9ef931f579 hooks/check_unittests: add proper intergration tests with mocking
Reviewed By: StanislavGlebik

Differential Revision: D14323720

fbshipit-source-id: 742e3db48d2ed7f767575585e319af2d7e79ff59
2019-03-11 07:11:41 -07:00
Lukas Piatkowski
b6d390acc1 hooks/check_unittests: use the intergraph endpoint result to make an accept/reject decision
Summary: This endpoint is also used in Mercurial now.

Reviewed By: StanislavGlebik

Differential Revision: D14303557

fbshipit-source-id: fe38b62d010de2846dcf800f93ba050d9c396873
2019-03-06 07:50:56 -08:00
Lukas Piatkowski
4e8ca0cb7e hooks: extract phabricator message parsing into convienient module
Summary: The phabricator message parsing capability will be also used from check_unittests Rust hook, so it had to be extracted and adjusted to Rust standards.

Reviewed By: StanislavGlebik

Differential Revision: D14301561

fbshipit-source-id: 47b59527dfadd7b761f750825da52ffce14fdf21
2019-03-05 10:32:30 -08:00
Stanislau Hlebik
6ddf4d16eb mononoke: revert D14143772
Summary:
Running hook test locally results in flaky errors - the new output just
disappears.

This is the error I'm getting

```
ERROR: test-hook-deny-files.t output changed
 --- test-hook-conflict-markers.t
+++ test-hook-conflict-markers.t.err
@@ -70,18 +70,6 @@
   searching for changes
   remote: Results of running hooks
   remote:   948f2cea 1
-  remote:     FAILED file hooks:
-  remote:       - conflict_markers on 1: Conflict markers were found in file '1'
-  remote:     no changeset hooks to run
-  remote:     1 of 1 file hooks failed
-  remote:     REJECTED
-  remote: Command failed
-  remote:   Error:
-  remote:     hooks failed
-  remote:   Root cause:
-  remote:     ErrorMessage {
-  remote:         msg: "hooks failed"
-  remote:     }
   abort: stream ended unexpectedly (got 0 bytes, expected 4)
   [255]

```

Apparently the output is not flushed properly

Reviewed By: HarveyHunt

Differential Revision: D14163914

fbshipit-source-id: b2acda3514ada0f720444f32f93997e148eed5f6
2019-02-21 05:47:31 -08:00
Lukas Piatkowski
23cf9d0f4f hooks: pretty print summary of running a hook during push
Reviewed By: StanislavGlebik

Differential Revision: D14143772

fbshipit-source-id: c7770dc57dc1d49c373e7ddfc5cfb05e708c36af
2019-02-20 08:52:09 -08:00
Lukas Piatkowski
49f5a481a3 hooks: initial diff for check_unittests with the outline of the hook
Summary:
In this first diff check_unittests doesn't do much except it calls the interngraph endpoint with the proper app id and auth token.
Mocking and proper logic of this hook will be added in following diffs.

Reviewed By: StanislavGlebik

Differential Revision: D14089265

fbshipit-source-id: 17c4330691ba3ffde33668e59f8b1cad4a9242a1
2019-02-15 13:16:27 -08:00
Lukas Piatkowski
8b3d09d620 hooks: enable configuring hooks for all bookmarks matching a given regex
Reviewed By: StanislavGlebik

Differential Revision: D14047223

fbshipit-source-id: 084b1843c10fedc429b16ab15e3bfc331e88d7ee
2019-02-13 09:07:18 -08:00
David Budischek
79cbf5c245 post-commit send scribe message
Summary: New commits should be logged to scribe, these will be used to trigger the update for the hg clone streamfile.

Reviewed By: lukaspiatkowski

Differential Revision: D14022599

fbshipit-source-id: a8a68f12a8dc1e65663d1ccf1a5eafa54ca2daf0
2019-02-13 07:26:33 -08:00
Kostia Balytskyi
7919782c14 mononoke: add bundle2_replay_params config option
Summary: This is to tell Mononoke whether to preserve bundle2 raw content or not.

Reviewed By: StanislavGlebik

Differential Revision: D14038588

fbshipit-source-id: f1781ed8b4aca7e37925e7f40aeb506af60d071a
2019-02-12 13:03:48 -08:00
Lukas Piatkowski
f6b3784baa hooks: use lower level API of blobrepo to access file type and size without fetching content
Summary: This should be a big improvement for hooks that only care about file type or size and not the content.

Reviewed By: StanislavGlebik

Differential Revision: D14023307

fbshipit-source-id: 28856512b3092af9517079a9f379d23a9c4f3864
2019-02-12 04:37:39 -08:00
Lukas Piatkowski
9c7f352b81 hooks tests: pass mocked reviewers to test-verify_reviewedby_info.t
Reviewed By: StanislavGlebik

Differential Revision: D13941629

fbshipit-source-id: b242b1465cd36cd85c80b9f69f4333058a5fbb5a
2019-02-08 07:39:56 -08:00
Lukas Piatkowski
4fe4f745fc hooks tests: pass mocked email values to test-hook-ensure-valid-email.t
Differential Revision: D13925110

fbshipit-source-id: 9bf82e56d8653fa60e46cab31937113f93ff54fa
2019-02-08 07:39:56 -08:00
Lukas Piatkowski
c163368a07 blobrepo/src/file.rs: use HgFileNodeId instead of HgNodeHash and propagate it
Summary: HgFileNodeId is a stronger typed id, so it is prefered to use it instead of HgNodeHash whenever it is identifying a filenode

Reviewed By: aslpavel

Differential Revision: D13986172

fbshipit-source-id: c0334652345acb868e86c38b8c0045e9c023c176
2019-02-07 04:04:28 -08:00
Lukas Piatkowski
515a2909eb monononoke hashes: remove usages of borrows of hashes which are Copy
Summary: The Copy trait means that something is so cheap to copy that you don't even need to explicitly do `.clone()` on it. As it doesn't make much sense to pass &i64 it also doesn't make much sense to pass &<Something that is Copy>, so I have removed all the occurences of passing one of ouf hashes that are Copy.

Reviewed By: fanzeyi

Differential Revision: D13974622

fbshipit-source-id: 89efc1c1e29269cc2e77dcb124964265c344f519
2019-02-06 15:11:35 -08:00
Lukas Piatkowski
44b06df35f hooks: do not pass repo name to hook execution
Summary:
Repo name is used only be verify_integrity hook and even there the name that Mononoke provides is incorrect. Instead of Mononoke's `repo-RepositoryId(1234)` name the hook is interested in Mercurial's `fbsource` name.
HookConfig is a perfect way to pass such an arbitrary parameter so use it.

Reviewed By: StanislavGlebik

Differential Revision: D13964486

fbshipit-source-id: 94090e409d5206828364202ae62a37abc16e4a27
2019-02-06 04:06:19 -08:00
Lukas Piatkowski
0e541063c1 hooks: pass configs down to Lua hooks via ctx
Reviewed By: StanislavGlebik

Differential Revision: D13896436

fbshipit-source-id: b610909cc20fdef8b91ccfd5efb7e2faed9c4d23
2019-02-01 05:52:20 -08:00
Lukas Piatkowski
adf0e95781 hooks: clean up lua hooks code to prepare for further changes
Summary:
This diff includes a bit of cleanup in hooks code, like:
- removing not used `repo_name` from `ctx.info`
- moving lua code from within `.rs` files to it's own `.lua` files
- addressing linter comments on lua code and
- reducing some copy-pasta between Changeset and File hook types

Reviewed By: StanislavGlebik

Differential Revision: D13895963

fbshipit-source-id: c867e060ad40e81fe0c81f30d14df59e75e129e6
2019-02-01 05:52:20 -08:00
Johan Schuijt-Li
5fd2d5ae7a handle uncaught exceptions in AclChecker
Reviewed By: StanislavGlebik

Differential Revision: D13808604

fbshipit-source-id: b4d807754d5f935507bd7c787abdf80c9ecb8654
2019-02-01 03:51:34 -08:00
Johan Schuijt-Li
1742bd1a24 mononoke hooks
Summary: Format code before editing.

Reviewed By: StanislavGlebik

Differential Revision: D13896166

fbshipit-source-id: 9133b2dddf8da6fff42cc7d5e6a71aecd9b18d7d
2019-01-31 06:35:13 -08:00
Stanislau Hlebik
3966192f01 mononoke: split hooks crate
Summary:
The main reason for doing it is to remove dependency on `BlobRepo` from hooks. Most of the  `hooks` crate code needs from `BlobRepo` just a HgBlobChangeset type, which was moved to a separate crate in one of the previous diffs. There is just a small piece of code that depends on blobrepo, and it was moved in the separate crate.

Because of that changing anything in BlobRepo won't trigger rebuilding of most of the hooks crate.

Reviewed By: lukaspiatkowski

Differential Revision: D13878208

fbshipit-source-id: d74336e959282c176258c653d4c408854e1f1849
2019-01-31 00:41:48 -08:00
Stanislau Hlebik
4d48415149 mononoke: split metaconfig crate
Summary:
Currently if a crate depends even on a single type from metaconfig then in
order to compile this trait buck first compiles metaconfig crate with all the
logic of parsing the configs.

This diff split metaconfig into two crates. The first one just holds the types for
"external consumption" by other crates. The second holds the parsing logic.

That makes builds faster

Reviewed By: jsgf, lukaspiatkowski

Differential Revision: D13877592

fbshipit-source-id: f353fb2d1737845bf1fa0de515ff8ef131020063
2019-01-31 00:41:48 -08:00
Lukas Piatkowski
9ff11fa3ef config: add HookConfig to pass configs in hashmap into hooks code
Summary: In this diff the configs are parsed from toml and passed around to hook's execution context. The actual usage of configs will be introduced in separate diff.

Reviewed By: StanislavGlebik

Differential Revision: D13862837

fbshipit-source-id: 60ac10aa9c25d224e703e1e55bef13dc481ba07e
2019-01-30 05:04:22 -08:00
Johan Schuijt-Li
8f2e32c7b8 Guard AclChecker::new and Identity::from_groupname as well from being included
Summary:
These calls make network requests which make our integration tests flaky, this
fix is a followup of D13580144

Reviewed By: StanislavGlebik

Differential Revision: D13580562

fbshipit-source-id: 110563c0360154a86cfc098a05240d7fd023f2c8
2019-01-04 08:32:29 -08:00
Johan Schuijt-Li
55fa57dbdf Disable AclChecker for integration tests
Summary:
Currently some tests are failing intermittently due to a timeout, this timeout
is caused by the integration tests making an external call to AclChecker which
is intermittently very slow.

For our integration tests we disable this call since it's not part of the test
suite to stop our tests from being flaky.

Reviewed By: StanislavGlebik

Differential Revision: D13580144

fbshipit-source-id: 0c26bb14dd222b888ca2638319071f4d99eab6df
2019-01-04 06:19:23 -08:00
Stanislau Hlebik
ae77a7d64b mononoke: remove failing test
Reviewed By: HarveyHunt

Differential Revision: D13455632

fbshipit-source-id: b2582a985153a0a0460ef577daedf203d1bb5da4
2018-12-18 05:48:19 -08:00
Lukas Piatkowski
5d9a151d85 mononoke: pass CoreContext down to blobstore
Reviewed By: jsgf

Differential Revision: D13324220

fbshipit-source-id: af7a2a650338ea66e504ea0acae1d103af10f8e3
2018-12-04 11:40:15 -08:00
Stanislau Hlebik
3ae6daeb0c mononoke: use skiplist in getbundle
Reviewed By: jsgf, lukaspiatkowski

Differential Revision: D13169016

fbshipit-source-id: 59c567663680a99dd977e087f38374e77c72afd6
2018-11-29 08:19:30 -08:00
Stanislau Hlebik
9bc593e6fa mononoke: remove unused RepoType
Summary: We can't really serve Mononoke from a revlog repo. Let's remove it

Reviewed By: HarveyHunt

Differential Revision: D13175450

fbshipit-source-id: 41ad328564bb0965bb93bf214bd89a1d456b397d
2018-11-23 01:23:09 -08:00
Harvey Hunt
368b2119bc Add regex_match to hook functions
Summary:
Add a helper function for hooks that can be used to match a string to a regex.

Regex matching can either be called from ctx.regex_match(s, r) or file.path_regex_match(r)

This will later be used in the no_bad_filenames hook, but will be helpful for other
hooks too.

Reviewed By: purplefox

Differential Revision: D12941544

fbshipit-source-id: f8f73a767da5ae77f486c925b6adc1333a198c3f
2018-11-22 09:28:19 -08:00
Tim Fox
d64d9121c5 Allow hook manager cache params to be configured
Summary: ${title}

Reviewed By: StanislavGlebik

Differential Revision: D13082486

fbshipit-source-id: fbdffdbfca218199dc2ac61abfcb8887b299db6d
2018-11-22 03:30:17 -08:00
Tim Fox
c25675a220 ensure_valid_email hook
Summary: Implement $name

Reviewed By: StanislavGlebik

Differential Revision: D13024134

fbshipit-source-id: e5693c262d9c28fb8c54ce0961dc8254e58fe01f
2018-11-13 09:37:23 -08:00
Tim Fox
b7a8069530 verify_integrity hook
Summary: Port verify_integrity hook to Mononoke

Reviewed By: StanislavGlebik

Differential Revision: D12839290

fbshipit-source-id: a04cdb96e534ba61d359008bef3f327b1758f8ee
2018-11-13 09:37:23 -08:00
Tim Fox
dafbef1dc3 Integrate Aclchecker with hooks
Summary:
Integrate AclChecker into hooks.

Note, TODO: Create a new hipster group "reviewers"

Reviewed By: StanislavGlebik

Differential Revision: D10507735

fbshipit-source-id: 62c02bdd2e79b8b77ea118ed97bec4a8f48f01db
2018-11-13 05:41:49 -08:00