Commit Graph

27 Commits

Author SHA1 Message Date
Jan Mazur
8152776a35 bypassing hooks for members of scm
Summary: Members of `scm` hipster group will be able to push to mononoke bypassing hooks when `BYPASS_ALL_HOOKS` pushvar is passed.

Reviewed By: krallin

Differential Revision: D24477468

fbshipit-source-id: ac910bf27e5510e1975c4a7cd0bfeff5216da70e
2020-10-29 10:14:25 -07:00
Kostia Balytskyi
1710662e9e hook_tailer: be able to run on push-redirected commits
Summary:
This is to be able to run the hook tailer on fbsource commits, which had been
push-redirected from ovrsource.

Reviewed By: StanislavGlebik

Differential Revision: D24447814

fbshipit-source-id: bfb164ec45dd2d4d5c3c83f133b48000a71a50a3
2020-10-22 02:48:18 -07:00
Kostia Balytskyi
660937cf9d hooks: be aware of native vs push-redirected pushes
Summary:
This change adds some wiring to allow the hooks to treat native vs push-redirected pushes differently. This is needed because `deny-files` needs to block native pushes to `.ovrsource-rest` and `arvr-legacy`, while it needs to allow push-redirection into these directories.

The plan further is to change the actual hook body for `deny_files` to have different handling of the two cases.

Reviewed By: StanislavGlebik

Differential Revision: D24257454

fbshipit-source-id: 2f5931149115210aeeeebb3294a6512effd36350
2020-10-13 05:46:56 -07:00
Jan Mazur
88bf94aacb move interngraph token from disk to keychain service
Summary:
Interngraph token is already stored within keychain service. We should make use of that.

I'll need to remove related config option in convigerator in a separate diff.

Reviewed By: krallin

Differential Revision: D24015463

fbshipit-source-id: 9e8246e2cc252f0c42669140de7b50410a15709c
2020-10-02 06:56:49 -07:00
Mark Thomas
b2b895353f hook_tailer: add --exclude-merges to skip merge commits
Summary: Add `--exclude-merges` which will skip merge commits.

Reviewed By: HarveyHunt

Differential Revision: D22945598

fbshipit-source-id: 3c20cf049bbe15a975671e8792259b460356804a
2020-08-05 10:34:05 -07:00
Mark Thomas
57626bec98 hook_tailer: add --log-interval to log every N commits
Summary:
Add `--log-interval` to log every N commits, so that it can be seen to be
making progress in the logs.

The default is set to 500, which logs about once every 10 seconds on my devserver.

Reviewed By: HarveyHunt

Differential Revision: D22945599

fbshipit-source-id: 7fc09b907793ea637289c9018958013d979d6809
2020-08-05 10:34:05 -07:00
Simon Farnsworth
454de31134 Switch Loadable and Storable interfaces to new-style futures
Summary:
Eventually, we want everything to be `async`/`await`; as a stepping stone in that direction, switch some of the blobstore interfaces to new-style `BoxFuture` with a `'static` lifetime.

This does not enable any fixes at this point, but does mean that `.compat()` moves to the places that need old-style futures instead of new. It also means that the work needed to make the transition fully complete is changed from a full conversion to new futures, to simply changing the lifetimes involved and fixing the resulting compile failures.

Reviewed By: krallin

Differential Revision: D22164315

fbshipit-source-id: dc655c36db4711d84d42d1e81b76e5dddd16f59d
2020-06-25 08:45:37 -07:00
Lukas Piatkowski
8bba936e5f mononoke/permission_checker: introduce MembershipChecker and its first usage in hooks
Summary: The new MembershipChecker and PermissionChecker traits will generalize access to various permission/acl systems (like LDAP) and leave the implementation details hidden behind an object trait.

Reviewed By: StanislavGlebik

Differential Revision: D21067811

fbshipit-source-id: 3bccd931f8acdb6c1e0cff4cb71917c9711b590b
2020-04-23 03:44:09 -07:00
Thomas Orozco
c12037e4af mononoke/hook_tailer: make concurrency configurable
Summary: That is helpful when e.g. benchmarking on the most humongous commits.

Reviewed By: farnz

Differential Revision: D21064716

fbshipit-source-id: 62973d8e4f0352a2d963bb2e8a87bdced6dedc85
2020-04-17 04:52:28 -07:00
Thomas Orozco
805a150bb6 mononoke/hook_tailer: support passing a list of changesets to tail
Summary:
This makes it easier to test performance on a specific set of commits. As part
of that, I've also updated our file reading to be async since why not.

Reviewed By: farnz

Differential Revision: D21064609

fbshipit-source-id: d446ab5fb5597b9113dbebecf97f7d9b2d651684
2020-04-17 04:52:28 -07:00
Thomas Orozco
d6d5129fa3 mononoke: add smoke tests for the hook tailer
Summary:
Let's try and make sure this doesn't bitrot again by adding a smoke test. Note
that there are no hooks configured here, so this exercises everything but the
actual hook running, but for now this is probably fine.

Note that this required updating the hook tailer to use the repository config
for the hook manager, since you can't start AclChecker in a test otherwise.

Reviewed By: StanislavGlebik

Differential Revision: D21063378

fbshipit-source-id: c7336bc883dca2722b189449a208e9381196300e
2020-04-17 04:52:27 -07:00
Thomas Orozco
1e28a37e0c mononoke/hook_tailer: remove a bit more dead code
Summary: What it says in the title

Reviewed By: farnz

Differential Revision: D21043171

fbshipit-source-id: 151a49cc0847b1b4f577df631c3cc6bb5ebfa77e
2020-04-17 04:52:26 -07:00
Thomas Orozco
53cb9829c6 mononoke/hook_tailer: stream outcomes
Summary:
Rather than buffer everything, let's stream outcomes as we go. Also, let's
track the number of changesets we accepted or rejected, as opposed to the hook
instance count (my goal is to output all that in a CSV if we want more detail).

Reviewed By: StanislavGlebik

Differential Revision: D21043173

fbshipit-source-id: 1b20339a52ac95a0a771b9ef469d19dd14ffc2c3
2020-04-17 04:52:26 -07:00
Thomas Orozco
b9bc56ada5 mononoke/hook_tailer: asyncify everything that's left
Summary: As it says in the title

Reviewed By: farnz

Differential Revision: D21042082

fbshipit-source-id: 0d5fb63ab380aa53a04352a8d8a474390127f68c
2020-04-16 02:15:24 -07:00
Thomas Orozco
e58a0868d5 mononoke/hook_tailer: remove continuous mode
Summary:
We don't use this anymore (instead we just do backtesting in bulk). Let's get
rid of it.

Reviewed By: farnz

Differential Revision: D21042083

fbshipit-source-id: af5aea3033a4d58ba61b8f22d7dc1249a112933e
2020-04-16 02:15:23 -07:00
Thomas Orozco
2ecf51e7af mononoke/hook_tailer: asyncify run_with_limit
Summary:
I'd like to clean up this code a little bit since I'm going to make a few
changes and would like to avoid mixing too many old and new futures.

Reviewed By: farnz

Differential Revision: D21042081

fbshipit-source-id: d6a807ce9c60d09d82c6b8c6866ea23b8ef45f21
2020-04-16 02:15:23 -07:00
Thomas Orozco
d37d58b51d mononoke/hook_tailer: remove dead code
Summary:
run_in_range isn't being used anywhere. Let's get rid of it. Also, let's not
make run_in_range0 a method on Tailer since it's more of a helper function.

Reviewed By: farnz

Differential Revision: D21042084

fbshipit-source-id: 2678a94ce4b0b6ae1c97e47eb02652bcbf238b0d
2020-04-16 02:15:23 -07:00
Thomas Orozco
10b815e1eb mononoke/hook_tailer: remove redundant roundtrip through hg cs id
Summary: What it says in the title.

Reviewed By: farnz

Differential Revision: D21042080

fbshipit-source-id: c5dbcc6179d01da2748d18ecae5b737c436e68a9
2020-04-16 02:15:22 -07:00
Thomas Orozco
a327fcb460 mononoke/hook_tailer: use csid_resolve
Summary: It's nice to be able to use a Bonsai ID if that's what you have.

Reviewed By: farnz

Differential Revision: D21040109

fbshipit-source-id: 4dfc447437053f9d7f4a1c9b3753d51fe5d02491
2020-04-16 02:15:22 -07:00
Simon Farnsworth
f8cc1c6e97 Delete HgChangeset hook handling completely
Summary: Not in use any more - all hooks are now Bonsai form - so remove it.

Reviewed By: krallin

Differential Revision: D20891164

fbshipit-source-id: b92f169a0ec3a4832f8e9ec8dc9696ce81f7edb3
2020-04-11 04:26:37 -07:00
Simon Farnsworth
da7cbd7f36 Run Bonsai hooks as well as old-style hooks
Summary: Running on Mercurial hooks isn't scalable long term - move the consumers of hooks to run on both forms for a transition period

Reviewed By: krallin

Differential Revision: D20879136

fbshipit-source-id: 4630cafaebbf6a26aa6ba92bd8d53794a1d1c058
2020-04-08 08:59:00 -07:00
Simon Farnsworth
b66d875fa5 Move hooks over from an internal representation based on HgChangesets to BonsaiChangesets
Summary: We want all hooks to run against the Bonsai form, not a Mercurial form. Create a second form of hooks (currently not used) which acts on Bonsai hooks. Later diffs in the stack will move us over to Bonsai only, and remove support for Mercurial changeset derived hooks

Reviewed By: krallin

Differential Revision: D20604846

fbshipit-source-id: 61eece8bc4ec5dcc262059c19a434d5966a8d550
2020-04-08 08:59:00 -07:00
Simon Farnsworth
d2cba5a169 Centralise hook running in the HookManager
Summary: We had hooks logic scattered around the place - move it all into the hooks crate, so that it's easier to refactor to use Bonsai changesets instead of hg.

Reviewed By: StanislavGlebik

Differential Revision: D20198725

fbshipit-source-id: fb8bdc2cdbd1714c7181a5a0562c1dacce9fcc7d
2020-03-18 09:17:17 -07:00
Simon Farnsworth
a908be34b3 Modernise hooks support
Summary: Migrate hooks to new futures and thus modern tokio. In the process, replace Lua hooks with Rust hooks, and add fixes for the few cases where Lua was too restrictive about what could be done.

Reviewed By: StanislavGlebik

Differential Revision: D20165425

fbshipit-source-id: 7bdc6820144f2fdaed653a34ff7c998913007ca2
2020-03-18 09:17:17 -07:00
David Tolnay
fe65402e46 rust: Move futures-old rdeps to renamed futures-old
Summary:
In targets that depend on *both* 0.1 and 0.3 futures, this codemod renames the 0.1 dependency to be exposed as futures_old::. This is in preparation for flipping the 0.3 dependencies from futures_preview:: to plain futures::.

rs changes performed by:

```
rg \
    --files-with-matches \
    --type-add buck:TARGETS \
    --type buck \
    --glob '!/experimental' \
    --regexp '(_|\b)rust(_|\b)' \
| sed 's,TARGETS$,:,' \
| xargs \
    -x \
    buck query "labels(srcs,
        rdeps(%Ss, fbsource//third-party/rust:futures-old, 1)
        intersect
        rdeps(%Ss, //common/rust/renamed:futures-preview, 1)
    )" \
| xargs sed -i 's/\bfutures::/futures_old::/'
```

Reviewed By: jsgf

Differential Revision: D20168958

fbshipit-source-id: d2c099f9170c427e542975bc22fd96138a7725b0
2020-03-02 21:02:50 -08:00
Lukasz Piatkowski
542d1f93d3 Manual synchronization of fbcode/eden and facebookexperimental/eden
Summary:
This commit manually synchronizes the internal move of
fbcode/scm/mononoke under fbcode/eden/mononoke which couldn't be
performed by ShipIt automatically.

Reviewed By: StanislavGlebik

Differential Revision: D19722832

fbshipit-source-id: 52fbc8bc42a8940b39872dfb8b00ce9c0f6b0800
2020-02-11 11:42:43 +01:00
Lukasz Piatkowski
e8d62b64d5 mononoke: move the codebase under eden/ directory
fbshipit-source-id: 43a0252cb3ec42aa365f20d1b6faa4d24d74c9b8
2020-02-06 13:46:04 +01:00