Summary:
Pushrebase should send back the newly created commits. This diff adds this
functionality.
Note that it fetches both pushrebased commit and current "onto" bookmark.
Normally they should be the same, however they maybe different if bookmark
suddenly moved before current pushrebase finished.
Reviewed By: lukaspiatkowski
Differential Revision: D9635433
fbshipit-source-id: 12a076cc95f55b1af49690d236cee567429aef93
Summary: We are going to use it in pushrebase as well
Reviewed By: lukaspiatkowski
Differential Revision: D9635432
fbshipit-source-id: 5cbe0879d002d9b6c21431b0938562357347a67f
Summary:
`asynchronize` does two conceptually separate things:
1. Given a closure that can do blocking I/O or is CPU heavy, create a future
that runs that closure inside a Tokio task.
2. Given a future, run it on a new Tokio task and shuffle the result back to
the caller via a channel.
Split these two things out into their own functions - one to make the future,
one to spawn it and recover the result. For now, this is no net change - but
`spawn_future` is likely to come in useful once we need more parallelism than
we get from I/O alone, and `closure_to_blocking_future` at least signals intent
when we allow a long-running function to take over a Tokio task.
Reviewed By: jsgf
Differential Revision: D9635812
fbshipit-source-id: e15aeeb305c8499219b89a542962cb7c4b740354
Summary:
`asynchronize` currently does not warn the event loop that it's
running blocking code, so we can end up starving the thread pool of threads.
We can't use `blocking` directly, because it won't spawn a synchronous task
onto a fresh Tokio task, so your "parallel" futures end up running in series.
Instead, use it inside `asynchronize` so that we can pick up extra threads in
the thread pool as and when we need them due to heavy load.
While in here, fix up `asynchronize` to only work on synchronous tasks and
push the boxing out one layer. Filenodes needs a specific change that's
worth extra eyes.
Reviewed By: jsgf
Differential Revision: D9631141
fbshipit-source-id: 06f79c4cb697288d3fadc96448a9173e38df425f
Summary:
We have suspect timings in Mononoke where `asynchronize` is used to
turn a blocking function into a future. Add a test case to ensure that
`asynchronize` itself cannot be causing accidental serialization.
Reviewed By: jsgf
Differential Revision: D9561367
fbshipit-source-id: 14f03e3f003f258450bb897498001050dee0b40d
Summary: In case `max_depth=1` we should only return the topmost entry, which in this case always is the root-entry. This fixes it so that we always return-fast in case `max_depth=1`.
Reviewed By: StanislavGlebik
Differential Revision: D9614259
fbshipit-source-id: a6b82bd5aac74d004f61a07bc24f5d26e5c56412
Summary: The latest release of `tokio` updates `tokio::timer` to include a new `Timeout` type and a `.timeout()` method on `Future`s. As such, our internal implementation of `.timeout()` in `FutureExt` is no longer needed.
Reviewed By: jsgf
Differential Revision: D9617519
fbshipit-source-id: b84fd47a3ee4fc1f7c0a52e308317b93f28f04da
Summary: While I was working on `actix-srserver`, I realized the current design of the API server is quite unnecessary. The "MononokeActor" and "MononokeRepoActor" are only returning futures without much CPU computation cost. So it don't need to be placed in a separate thread.
Reviewed By: jsgf
Differential Revision: D9472848
fbshipit-source-id: 618ec39c42d90717fa6985fee7d6308420962d3f
Summary: It avoids a heap of copies
Reviewed By: StanislavGlebik
Differential Revision: D9595689
fbshipit-source-id: a64f0a383acd517830d08cf0be9fc0a1b6903382
Summary:
Looks like we shouldn't have raised in the first place. Big getfiles buffer
causes OOMs on the servers. Also memory profiling shows that quite often most
of the Mononoke server's memory is used for serving remotefilelog requests.
Reviewed By: purplefox
Differential Revision: D9601990
fbshipit-source-id: 356a65d0749b064486436fb737bd5a47b3beecfa
Summary:
Add the ability to get bookmarks using the mononoke admin tool.
Usage: `mononoke_admin --repo-id <repo-id> bookmarks get --changeset-type <hg|bonsai> <BOOKMARK_NAME>`
The changeset-type defaults to HG.
Reviewed By: StanislavGlebik
Differential Revision: D9556742
fbshipit-source-id: c5e64981947aabb9059295622501bc359ed57cc6
Summary:
Add the ability to set bookmarks using the mononoke admin tool.
Usage: `mononoke_admin --repo-id <repo-id> bookmarks set <BOOKMARK_NAME> <HG_CHANGESET_ID>`
Reviewed By: StanislavGlebik
Differential Revision: D9539550
fbshipit-source-id: 7114a6a51711eae6784eb30d820c2ce11672679c
Summary:
Pushkey parts can be sent as part of pushrebase. Phases pushkeys are ignored
because we don't yet support them in Mononoke.
It's not really clear what to do with bookmark pushkey parts, since we are
already moving the `onto` bookmark. For now I suggest to ignore moves of the
`onto` bookmark, and error if there is a pushkey of another bookmark.
Reviewed By: farnz
Differential Revision: D9554385
fbshipit-source-id: 07aff1bd9034c0f2d56a2a5a66ea33c91835ef98
Summary:
I was debugging pushrebase bugs, and the only error I got was
'oneshot::Cancelled'. Let's add a bit more context around it
Reviewed By: farnz
Differential Revision: D9554384
fbshipit-source-id: b3111ef1b5c743d65740f7fa3fd1a92eef9ab784
Summary:
This diff fills missing parts of push-rebase implementation
- `find_closest_root` - find closest root to specified bookmark
- `find_changed_files` - find file affected by changesets between provided `ancestor` and `descendant`
- `intersect_changed_files` - rejects push rebase if any conflicts have been found
- `create_rebased_changes` - support for merges
- `do_pushrebase` - returns updated bookmark value
Reviewed By: StanislavGlebik
Differential Revision: D9458416
fbshipit-source-id: c0cb53773eba6e966f1a5928c43ebdec761a78d3
Summary:
Modify the lookup() RPC function to be able to accept either a
bookmark or commit hash. A commit hash lookup is attempted first, falling
back to a bookmark lookup if it fails.
Reviewed By: StanislavGlebik
Differential Revision: D9457349
fbshipit-source-id: 78db21c01c498b045f5781097cb12f7220a40999
Summary: Added a thrift client library and binary for Mononoke API Server that allows us to play with the API Server's thrift port.
Reviewed By: farnz
Differential Revision: D9110899
fbshipit-source-id: 603cc5e2b5e0419a73c9eccb35f8c95455ada9ce
Summary: This commit adds a basic thrift server that responds to fb303 status check queries to Mononoke API Server.
Reviewed By: farnz
Differential Revision: D9092291
fbshipit-source-id: d1e4ddb280c252f549d40a0bb03d05afccbf73b8
Summary: This commits change `HgBlob` from an enum into a struct that only contains one Bytes field, completely removes `HgBlobHash` and changes the methods of `HgBlob` from returning `Option`s into directly returning results.
Reviewed By: farnz
Differential Revision: D9317851
fbshipit-source-id: 48030a621874d628602b1c5d3327e635d721facf
Summary: Since this data is specific to TimedStream and not TimedFuture I split the Stats struct into FutureStats and StreamStats
Reviewed By: StanislavGlebik
Differential Revision: D9355421
fbshipit-source-id: cc2055706574756e2e53f3ccc57abfc50c3a02ba
Summary: gettreepack doesn't care for deleted entries, only about added or modified ones
Reviewed By: StanislavGlebik
Differential Revision: D9378909
fbshipit-source-id: 2935e6b74fbb0208f7cf89ab4b1e761bb9c6000b
Summary: Now pushrebasing stacks as well. Again, still no conflicts checks
Reviewed By: aslpavel
Differential Revision: D9359807
fbshipit-source-id: 9f6e7a05b45fb80b40faaaaa4fe2434b7a591a7c
Summary:
Revsets must use ChangesetId, not HgNodeHash. I'm going to use
`RangeNodeStream` in pushrebase so I thought it was a good time to change it
Reviewed By: farnz
Differential Revision: D9338827
fbshipit-source-id: 50bbe8f73dba3526d70d3f816ddd93507db99be5
Summary:
Initial implementation of pushrebase. At the moment it processes just one commit, but after implementing stub function it should work for every case.
Note that there is a special PushrebaseError. This is necessary to distinguish between infra error (i.e. mysql is unavailable) and expected errors like conflicts.
Reviewed By: aslpavel
Differential Revision: D9306815
fbshipit-source-id: 7c3f91b17c6270537d63e8c9dba8116f96840ece
Summary: Just failing for now, next diffs will add an actual functionality
Reviewed By: farnz
Differential Revision: D9306814
fbshipit-source-id: c515f2e742833833d73bce08dbea1ddbb7e2ae79
Summary:
First step of implementing pushrebase algorithm. Save the commits that client
has sent us. The parts that client sends us are the same as in normal push
except for the names and parameters.
Reviewed By: farnz
Differential Revision: D9304750
fbshipit-source-id: d5be6635c0cf1a14a66a5fed5ba13f344195e8bc
Summary:
Sometimes the scribe writes can fail due to backpressure, so just drop
them while still logging to stdout.
Reviewed By: StanislavGlebik
Differential Revision: D9355416
fbshipit-source-id: 8cebe61b1ccfe802fcff686102096d1c9291aa1a
Summary:
Added some comments and fixed a couple of little style issues.
Log when warmup prefetching starts and ends.
Reviewed By: StanislavGlebik
Differential Revision: D9355414
fbshipit-source-id: b16ac267cc0abda01ab445ca3e5de34c17f680a7
Summary: No need for a whole file for a single use statement.
Reviewed By: StanislavGlebik
Differential Revision: D9349613
fbshipit-source-id: 511985201e0799a0c4f0847d14a7c439fa249687
Summary:
Caught by inspection - this builds up a long linked list of streams
to evaluate, where `select_all` has a vector and uses `FuturesUnordered` to
optimize which stream to check next.
Reviewed By: lukaspiatkowski
Differential Revision: D9359932
fbshipit-source-id: 83e645cc5f17a6376d0f0925b88cb390caf3dc4f
Summary:
Rebalancing causes cachelib to do early eviction of a slab in order to
increase hit rate in the cache as a whole. Now that we have much larger caches,
doing this infrequently makes sense - increase from every 10 seconds to every 5
minutes.
This will decrease the frequency of rebalancing logs
Reviewed By: StanislavGlebik
Differential Revision: D9343958
fbshipit-source-id: 42a33ac310083585318c1fc503bf0b1717b9fb61
Summary:
It makes startup unbearably slow, and doesn't add any benefits at all. Revert
it
Reviewed By: purplefox
Differential Revision: D9358741
fbshipit-source-id: 26469941304f737c856a6ffca5e577848ad30955
Summary:
We need to look for bonsai changeset only if it exists, otherwise blobimport
will just fail
Reviewed By: aslpavel, farnz
Differential Revision: D9358689
fbshipit-source-id: fc42fbe161670d46e1fb46ef2bea98ad9faea0a2
Summary: This is more general, and allows one to call `RepoClient` methods.
Reviewed By: farnz
Differential Revision: D9318658
fbshipit-source-id: 09b2e64bc0d423eafcb381902e03f349fc666a41
Summary: ugh, yet another case of a hidden dependency.
Reviewed By: StanislavGlebik
Differential Revision: D9318498
fbshipit-source-id: 5fcd25081b5033cbef9c5f137e616348c5d6ced9
Summary: Remove `Arc<BlobRepo>` from more places since `BlobRepo` will do its own internal `Arc`ing.
Reviewed By: StanislavGlebik
Differential Revision: D9317987
fbshipit-source-id: 899e8b2ede278e62a83e64c144eb18c8cc7e57c6
Summary: Adds proper url decoding for is_ancestor, so that special characters can be encoded in the url.
Reviewed By: kulshrax
Differential Revision: D9325467
fbshipit-source-id: d3ff60e004be8d254ea6f7288188adf54ab7ff5f
Summary: Beginnings of a container with various essential bits.
Reviewed By: StanislavGlebik
Differential Revision: D9322148
fbshipit-source-id: b69bd17aa88acd69e81b90e9a1efb672247dc887
Summary:
It's really part of the server, and isn't likely to be useful
elsewhere.
Reviewed By: StanislavGlebik
Differential Revision: D9322149
fbshipit-source-id: 0dc3ca41f2779b3cc9e1c32f8e09e369038c3d53
