Summary:
Deleted the RepoGenCache structure, associated file, and public exports.
Also deleted the containing repoinfo crate, as nothing else was using it now.
Deleted some existing references to it in experimental code which weren't caught in the test plan but were blocking this from landing.
Reviewed By: StanislavGlebik
Differential Revision: D8787103
fbshipit-source-id: 0b90c758ea8175cb0f3ec74c371592b9ca5b192e
Summary: Removed RepoGenCache from RangeNodeStream structure, methods, and tests. Last step is to delete RepoGenCache structure.
Reviewed By: StanislavGlebik
Differential Revision: D8786943
fbshipit-source-id: 70fa5b82da548d11126a289eb2cb11aaace18463
Summary: Completely removed references to RepoGenCache in the test helper function assert_node_sequence, and updated all usages of it in the revset tests.
Reviewed By: StanislavGlebik
Differential Revision: D8786191
fbshipit-source-id: f1504e7a95e5555c86a2f02cb98ce1f28e374eab
Summary: Replaced usage of a RepoGenCache in a revset test helper with repo.get_generation() calls. The RepoGenCache parameter is still passed, but is now unused and marked as such. Next step is to remove it from all of the tests and as a parameter altogether.
Reviewed By: StanislavGlebik
Differential Revision: D8785579
fbshipit-source-id: 95ec575117a7dfea5d683a683d54d19de2835a3b
Summary: Fixed a bug in which AncestorNodeStream was excessively slow if it were used in cache_warmup. Changing how a node hash was mapped to its parents made this a lot faster.
Reviewed By: farnz
Differential Revision: D8754990
fbshipit-source-id: bbd0cd97b516b7d3a5f9e048c7bdea0f24d6f7f3
Summary:
Removed all references to RepoGenCache from publically callable functions in the revset package. This involved:
- Modifying blobrepo so that its get_generation_number method returned a Generation wrapper instead of a raw usize, to allow it to be used in a cleaner manner in the revset code.
- Simultaneously changing the constructors of all the structures in revset. This seems like a big change, but many of them call each other, passing a RepoGenCache object down the line, so eliminating them all at once made for the cleanest update.
- Modifying helper functions in the revset structures which would create streams of nodes by taking ownership of a RepoGenCache object within a closure. Instead they now take ownership of a clone of the repo. This strategy was already done earlier in the same helper functions, so I am assuming the cost of cloning a repo into a closure is small.
- Modifying the only external usage of revset within the mononoke server code.
This is part of a several step process to completely remove RepoGenCache from the code base. The next steps should be:
- Remove all references to RepoGenCache in the testing macros for revset.
- Delete RepoGenCache and clean up any dangling references to it.
Reviewed By: StanislavGlebik
Differential Revision: D8743560
fbshipit-source-id: 125f851075d836d40224d339e1daee912a39f7e4
Summary:
The CachingBlobstore can use a LeaseOps implementation to prevent
multiple parallel writes to the same backing store. Provide a dummy implementation
that does not block parallel writes, for performance testing and to use where
there's no way to implement a useful lease operation using a given cache.
Reviewed By: StanislavGlebik
Differential Revision: D8809077
fbshipit-source-id: 54cf67882a42b43dde1c6ac68611fb28144509bc
Summary:
- [moononoke][bonsai_hg_mapping] id for unique `(repo_id, hg_cs_id)` in mysql schema
- NOTE: sqlite does not understand this syntax so it is left unchanged
Reviewed By: StanislavGlebik
Differential Revision: D8820198
fbshipit-source-id: f0e2478900dc68d619aef0ed16602b76371c85f8
Summary:
Those are the changes made to crates in tp2: P59806629
Several changes that were hard to split are here:
- update manifold client to use hyper 0.12.x
- update scribe client to use hyper 0.12.x
- update mononoke manifoldblob to use the updated manifold client
- update mononoke hgcli to use openssl 0.10.x and tokio-openssl
- update mononoke server to use openssl 0.10.x and tokio-openssl
- remove sendwrapper
Reviewed By: jsgf
Differential Revision: D8806931
fbshipit-source-id: 65412d483f77d8c0a0d5692c41c6516bb8f86046
Summary: This will be useful for the bonsai verification tool.
Reviewed By: StanislavGlebik
Differential Revision: D8792562
fbshipit-source-id: f409d0fa042528b04462a1539fd3c2a8064a4f6e
Summary: This code can easily be shared.
Reviewed By: StanislavGlebik
Differential Revision: D8777307
fbshipit-source-id: f11314f6a63bb191dc38d07cec181a4b05b158d9
Summary:
Creating the package for reachabilityindex, where methods that support efficient reachability queries on BlobRepo objects will live.
Includes the initial definition of the trait ReachabilityIndex which methods supporting reachability queries on BlobRepo objects will implement.
Reviewed By: StanislavGlebik
Differential Revision: D8792977
fbshipit-source-id: 293a810518e0a1fa260ccbc38902484e56ef2038
Summary:
We need to store relation between Hg changesets and Bonsai changesets.
- `BonsaiHgMapping` is exactly this mapping which establishes injective relation between `{Hg|Bonsai}Changeset`
Reviewed By: StanislavGlebik
Differential Revision: D8801254
fbshipit-source-id: c7df14172e6c2d67c039a24e1bb821e6d92860af
Summary: - integration test of were failing on my devserver as `{http|https}_proxy` environment variables
Reviewed By: StanislavGlebik
Differential Revision: D8804505
fbshipit-source-id: fafea55ed80897b24ad816bc26b241e64cd8b913
Summary:
The memcache implementations of CacheOps and LeaseOps are fairly complex, with lots of edge cases.
Add some stats, so we can see which edge cases are being hit, and optimize accordingly
Reviewed By: lukaspiatkowski
Differential Revision: D8752599
fbshipit-source-id: fe1376956beceff702c2b6a020598556016033b5
Summary:
When we add cachelib bindings to Rust, we're going to want to implement a
cachelib blobstore that's more or less the same as the memcache version, but
backed by a cachelib pool instead of a memcache instance.
Split this code up so that we don't duplicate functionality
Reviewed By: StanislavGlebik
Differential Revision: D8523713
fbshipit-source-id: 882298abab8c208103f6d8c74fee60a768c877f6
Summary:
This is a series of patches which adds Cargo.toml files to all the crates and tries to build them. There is individual patch for each crate which tells whether that crate build successfully right now using cargo or not, and if not, reason behind that.
Following are the reasons why the crates don't build:
* failure_ext and netstring crates which are internal
* error related to tokio_io, there might be an patched version of tokio_io internally
* actix-web depends on httparse which uses nightly features
All the build is done using rustc version `rustc 1.27.0-dev`.
Pull Request resolved: https://github.com/facebookexperimental/mononoke/pull/7
Differential Revision: D8778746
Pulled By: jsgf
fbshipit-source-id: 927a7a20b1d5c9643869b26c0eab09e90048443e
Summary: by using futures rather than reactor::run calls we are able to add f.e. tracing and timeouts easily - in the next diff
Reviewed By: farnz
Differential Revision: D8750662
fbshipit-source-id: d928d903cf5631a275f5b319d075db9f1611d76b
Summary:
hgcli will start logging stuff as well and it will use the same session_uuid as the server.
This also includes logging the user and source hostname.
Reviewed By: farnz
Differential Revision: D8750663
fbshipit-source-id: 7ebc8b6c10b7560d985fd23e9e3f2645f3bd0a1c
Summary: those structures are sshrelay specific, move them there so it's easier to share them in future diffs
Reviewed By: farnz
Differential Revision: D8750666
fbshipit-source-id: b58596e63787d221a3970d5f1648e11d81949925
Summary:
Unify all uses of Sqlite and of Mysql
This superceded D8712926
Reviewed By: farnz
Differential Revision: D8732579
fbshipit-source-id: a02cd04055a915e5f97b540d6d98e2ff2d707875
Summary: This is a preparation to using Preamble more heavily
Reviewed By: StanislavGlebik
Differential Revision: D8750665
fbshipit-source-id: 44d4bcedbe95fe05679faeedf4479ebad4d9359c
Summary: This commit bridges messages logged via `log` crate to be forwarded to `slog` using `slog-stdlog` crate. This allows us to see debug messages printed from third party crates that are using `log` instead of `slog`.
Reviewed By: StanislavGlebik
Differential Revision: D8703996
fbshipit-source-id: f847b76a52c262eded4f576a5d9065574ff7e4dd
Summary: This commit adds a scuba middleware for actix. It will automatically collects metrics from the request and the response.
Reviewed By: StanislavGlebik
Differential Revision: D8706387
fbshipit-source-id: 4dddad0d977d42718ce53e1e4fd7b771a1eda7df
Summary: This commit splits middleware.rs into mod middleware, and move logic of computing responsive to a separate trait so it can be shared.
Reviewed By: StanislavGlebik
Differential Revision: D8715657
fbshipit-source-id: cf06249d41b9fbe1a4a7500ed5e8024bbf376d3f
Summary: This diff refactors the hook code to use the Bookmark struct instead of strings
Reviewed By: farnz
Differential Revision: D8724197
fbshipit-source-id: 920aa1266ca94b2bd8683a995e4fd781159bd5b1
Summary: This diff updates rep config to support per file hooks. We also extend the hookloader to support per file hooks.
Reviewed By: lukaspiatkowski
Differential Revision: D8713464
fbshipit-source-id: 65cf20957506adc620bf06d84707669d125366ed
Summary:
This diff extends lua hooks to support per file hooks.
We also now call the hook function via a Lua wrapper function which allows us to have a better API in the hook as we can construct lua table to pass into the hook and we can also return multiple return values from the hook, which allows us to support rich hook failure reasons properly. Both of these are hard to do when calling hooks directly using hlua.
Reviewed By: StanislavGlebik
Differential Revision: D8711280
fbshipit-source-id: b91f9e47a1f8eab302775a5bbfd61590a8635282
Summary: Replaced the old behavior of getting the 'next_generation' via a max call on a current set of generation numbers, with a heap which stores the currently viewed set. If the currently viewed set gets large in practice, then this should have a significant speed up.
Reviewed By: farnz
Differential Revision: D8725928
fbshipit-source-id: e03f4573bb14e7170d23dbbb0fd62118761bec62
Summary: Moved the Generation struct, which is a u64 wrapper, from repogen to mononoke-types, and updated the according usages. This should make it easier to phase out RepoGenCache.
Reviewed By: farnz
Differential Revision: D8725538
fbshipit-source-id: cfd39be03bae56d2288053b7b5e212e6d547c833
Summary: Session UUID will help identify the issues on Mononoke side whenever the client encounters problems
Reviewed By: StanislavGlebik
Differential Revision: D8732396
fbshipit-source-id: 35d04b0d56be0cfc2c608f08287a2b1d236a96e3
Summary:
We've recently added a flag in a config repo to enable or disable the repo.
Let's use it in apiserver as well.
P. S.
As I think about it now, it maybe worth filtering enabled/disabled flag in
RepoConfig
Reviewed By: lukaspiatkowski
Differential Revision: D8732450
fbshipit-source-id: 1cd59f5a91dec28efd1e2c122f2f1ea58ded6ac1
Summary:
This diff implements hooks which act upon individual files. The results of the hooks are cached using Asyncmemo.
The cache is currently keyed on (changeset_id, hook_name, file path) but this will change to file content hash once we move to Bonsai changesets.
Reviewed By: StanislavGlebik
Differential Revision: D8707726
fbshipit-source-id: ceaf94abd09e1dd7f6b2d8f9c87a9a221439a252
Summary: This diff removes the unnecessary ffi in lua_hook and instead uses lua.into_get to get the hook function without borrowing.
Reviewed By: farnz
Differential Revision: D8525921
fbshipit-source-id: b1d43c919c2aac113ee58fde28043639e2bd2362
Summary: This diff introduces a hook loader, which has the responsibility of loading Lua hooks into the hook manager given a repo config.
Reviewed By: StanislavGlebik
Differential Revision: D8625177
fbshipit-source-id: 51948d9a70c02b859ab3a9b45c8487a9da6e7f6a
Summary: This diff implements tracking of hooks by bookmark in the hook manager and functionality to run hooks per bookmark.
Reviewed By: StanislavGlebik
Differential Revision: D8525598
fbshipit-source-id: 7987d1f8d90a77667f120f4940f12aa3cb5aa86e
Summary:
This diff refactors the server config repository to support storing and loading of hooks. In the new structure each repo lives in its own directory and the config file for the server is called "server.toml".
Hooks can be referenced by relative or absolute paths allowing either local or common hooks to be loaded.
Reviewed By: StanislavGlebik
Differential Revision: D8625178
fbshipit-source-id: 62c8c515a0fbbf7a38cfc68317300d8f42eb4d7a
Summary: Took the UniqueHeap struct that was made for diff D8500722 and moved it into its own file, anticipating refactoring other parts of revset to use it.
Reviewed By: StanislavGlebik
Differential Revision: D8724580
fbshipit-source-id: 0d71cbc81271912b044048181fc66b513269a9b2
Summary:
In case the bookmarks have been updated while we are reading the revlogs we shoud read the bookmarks before reading changestes.
In this diff we are also reading bookmarks after importing so that we ensure that the newest possible version of bookmarks are persisted.
Reviewed By: farnz
Differential Revision: D8723522
fbshipit-source-id: 278382dae10a0554abc8edc398e7a15a37569676
Summary:
Correctly resolve file/directory conflict in `MemoryManifest`
- when we have file/directory conflict and have entry in `BonsaiChangeset` that indicates that file should be deleted we need resolve conflict by keeping directory
- more details in [Bonsai changeset actions](https://fb.quip.com/A2kqArd9Nb90#WHBACA4Vitd)
Reviewed By: lukaspiatkowski
Differential Revision: D8658958
fbshipit-source-id: 815a91b847c5c0c9586eb5d317ebb9412196d5f4
Summary:
New parameter depth was added. Mononoke just ignores, which if correct, but can
be less efficient. However test-gettreepack.t was failing because it didn't
supply the parameter. This diff fixes it by sending a big depth to the server
P. S.
Attached task tracks the support of depth parameter on Mononoke server-side
Reviewed By: lukaspiatkowski
Differential Revision: D8712906
fbshipit-source-id: 8246344b3fa39f00eb39f2262b4aa5aa74834be2
Summary:
They were broken since 2646d41a64917d3e50f662b0b4b628ccfdbb05a8.
It added enabled_repo repository to config, however, no enabled_repo has ever been
blobimported. Since we don't really need enabled_repo, let's remove it
Reviewed By: farnz
Differential Revision: D8712891
fbshipit-source-id: 360f2c9564087d3a8411871335aa40fa81e38c0c
Summary:
We need to be able to distinguish logical transaction failure from
infrastructure failure, so change the `Transaction::commit` future to
`Future<Item=bool, Error=Error>`, where `false` indicates a logical transaction
failure. This allows the caller to determine whether a retry or other recovery
logic is needed.
Reviewed By: lukaspiatkowski
Differential Revision: D8555727
fbshipit-source-id: 8ab64f3019f2644e7eaabc8d699d99aa8eb08fbb
Summary:
Add a per-repo config flag to repos to be configed without being
enabled. Setting "enabled = false" will make Mononoke completely ignore the
repo config. If not present, "enabled" is assumed to be true.
Reviewed By: farnz
Differential Revision: D8647161
fbshipit-source-id: 2646d41a64917d3e50f662b0b4b628ccfdbb05a8
Summary:
Use tls for connection between hgcli and Mononoke server always, even for
localhost connections[1]
The setup is similar to tls setup of Eden server.
[1] This is not necessary of course, but adding an option to bypass tls
connection may result in accidental use of it in prod. However if it turns out
to be too unusable, we can add such option in the future
Reviewed By: jsgf
Differential Revision: D8644299
fbshipit-source-id: 0898e30e33b718e13a766763479f3adf9323ffe7
Summary: This commit upgraded openssl, enabled alpn for actix-web and added tokio-codec with fixes due to the upgrade.
Reviewed By: StanislavGlebik
Differential Revision: D8682673
fbshipit-source-id: 8c7cadfd6c0c7b016202f6cb038eb4951d0f9333
