Commit Graph

20 Commits

Author SHA1 Message Date
Stanislau Hlebik
d0eac0f60f mononoke: save bundles for replay
Summary:
Together with logging bookmark moves, let's also log bundle handle. It will be
used during replay from Mononoke to mercurial.

Reviewed By: ikostia

Differential Revision: D13990929

fbshipit-source-id: 4039322903b13e84fb31c8e65cc2e097ca765213
2019-02-11 05:50:55 -08:00
Stanislau Hlebik
a75a1453f7 mononoke: add bookmark reason moves
Summary:
This is the first step in adding support for tracking all bookmark moves. They
will be recorded in the separate mysql table in the same transaction as
bookmark is updated.

That gives us two things:
1) Ability to inspect all bookmark moves and debug issues with them
2) Also record which mercurial bundle moved a bookmark if any so that we could
later replay these bundles in correct order on hg

Add a struct that let us track bookmark moves.

Reviewed By: ikostia

Differential Revision: D13958872

fbshipit-source-id: 9adfee6d977457db5af4ad5d3a6734c73fcbcd76
2019-02-11 05:50:54 -08:00
Stanislau Hlebik
56a2ad455c mononoke: remove mysql schemas
Summary:
These are **not** the schemas that we use in production.
So at the moment they are not used for anything and they just create confusion.

Reviewed By: aslpavel

Differential Revision: D13986001

fbshipit-source-id: 7aae0a5da474f579c9cdf1bbf5dfe183835cae2d
2019-02-08 03:04:01 -08:00
Lukas Piatkowski
515a2909eb monononoke hashes: remove usages of borrows of hashes which are Copy
Summary: The Copy trait means that something is so cheap to copy that you don't even need to explicitly do `.clone()` on it. As it doesn't make much sense to pass &i64 it also doesn't make much sense to pass &<Something that is Copy>, so I have removed all the occurences of passing one of ouf hashes that are Copy.

Reviewed By: fanzeyi

Differential Revision: D13974622

fbshipit-source-id: 89efc1c1e29269cc2e77dcb124964265c344f519
2019-02-06 15:11:35 -08:00
Stanislau Hlebik
0b0a5b93b2 mononoke: extract blobrepo_factory and postcommit from blobrepo
Summary:
`blobrepo_factory` is a crate that knows how to create blobrepo given
a configuration i.e. it creates blobstores, filenodes, changesets etc and
initializes blobrepo with them.

`post_commit` is a small part of blobrepo which can also be extracted from
blobrepo crate.

There are a few upsides with this approach
1) Less dependencies on Blobrepo, meaning we have to rebuild it fewer times
2) BlobRepo compilation is faster

Reviewed By: jsgf

Differential Revision: D13896334

fbshipit-source-id: 1f5701341f01fcefff4e5f9430ddc914b9496064
2019-02-05 01:23:15 -08:00
Stanislau Hlebik
a6f6f28564 mononoke: split reachability index
Summary:
Let's split reachability index crate. The main goal is to reduce compilation
time. Now crates like revsets will only depend on traits definition but not on
the actual implementation (skiplist of genbfs).

Reviewed By: lukaspiatkowski

Differential Revision: D13878403

fbshipit-source-id: 022eca50ac4bc7416e9fe5f3104f0a9a65195b26
2019-01-31 00:41:48 -08:00
Stanislau Hlebik
3d06332693 mononoke: move changeset_fetcher out of blobrepo into separate crate
Summary:
Some crates, namely revsets and reachabilityindex, currently depend on
blobrepo, while all they need is the ability to fetch commits.

By moving changeset_fetcher outside this dependency will be removed. That may
make builds faster

Reviewed By: lukaspiatkowski

Differential Revision: D13878369

fbshipit-source-id: 9ee8973a9170557a4dede5404dd374aa4a000405
2019-01-31 00:41:48 -08:00
Liubov Dmitrieva
4be4e90187 optimization: add new api for phasing that allows to reuse bookmarks
Summary:
This will reduce max potential number of fetching bookmarks from O(len(longest stack)) to O(1) (with is 0 or 1) if there are draft commits.

Previously, the refetching could happen on every iteration while we are
searching for public roots.

This diff allows us to reuse the bookmarks from the first time they were needed
and has been fetched in the whole phases calculation process. Fetch just once!

If there are no draft commits bookmarks were fetched only once already (and only if needed).

Reviewed By: StanislavGlebik

Differential Revision: D13753520

fbshipit-source-id: d96a6cf434cb4a1fe95e51ae734afb1671124336
2019-01-25 10:14:42 -08:00
Liubov Dmitrieva
23b3931529 add phases calculation for public roots
Summary:
this is required to cover corner cases when client has some stacks and part of those became public

calculation for public roots happen for draft heads only, it doesn't change performance of hg pull

Reviewed By: StanislavGlebik

Differential Revision: D13742685

fbshipit-source-id: d8c8bc357628b9b513bbfad4a82a7220d143f364
2019-01-22 05:58:46 -08:00
Liubov Dmitrieva
fd7345bc58 calculate phases using bulk API
Summary:
The bulk api makes less queries to mysql and therefore is more efficient.

This is especially important for `hg pull` requests where the list of heads is very large.

Reviewed By: lukaspiatkowski

Differential Revision: D13677298

fbshipit-source-id: 3dec1b3462c520c11481325e82523ef7a6ae6516
2019-01-16 08:22:54 -08:00
Liubov Dmitrieva
74ed84b7c3 implement bulk phases fetch and set: part2
Summary:
the remaining part to implement bulk phases fetch and update

this is required to optimize number of MySql queries we are using to look up at the db for phases and bookmarks.

the single get api has been removed and reimplemented by calling bulk ones

Reviewed By: aslpavel

Differential Revision: D13664900

fbshipit-source-id: 29342e86c057b92e331fadcebe51f452d9569e09
2019-01-15 09:47:04 -08:00
Liubov Dmitrieva
be4f3857b1 implement bulk phases fetch and set: part1
Summary:
this is required to optimize number of MySql queries we are using to look up at the db for phases and bookmarks.

the next step is to add the same with Memcache to caching.rs

implementations for the single get is replaced with just calling the implementation for multiple get

Reviewed By: aslpavel

Differential Revision: D13658610

fbshipit-source-id: e3876044e2cbbefb156175c51ab7051db3885eb8
2019-01-15 09:11:21 -08:00
Liubov Dmitrieva
eeb303f932 using not the default skip index for phases calculations
Summary:
use the correct skip index

sorry for some rustfmt.

Reviewed By: StanislavGlebik

Differential Revision: D13636059

fbshipit-source-id: 2815d82b63b86bda053f5a3a9a1b8a3b72abbf82
2019-01-14 06:27:47 -08:00
Liubov Dmitrieva
8a138c33c8 add support for encording of phase-heads part of bundle2
Summary:
We have decided that this will be used to transport phases to the client

Hg client already supports this part.

Reviewed By: StanislavGlebik

Differential Revision: D13507921

fbshipit-source-id: 621e93bb6e1a0c87d4f4963ba7fa635b77a5b6ec
2019-01-08 05:32:36 -08:00
Liubov Dmitrieva
56dff9277a optimize phase calculation
Summary:
Basically if any of the bookmarks is reachable, all other bookmarks are not
interesting to check, so those futures should be skipped.

Reviewed By: StanislavGlebik

Differential Revision: D13538511

fbshipit-source-id: 193a7ea8d505690aeb96247a07c8f2688cd7a59f
2018-12-21 12:25:05 -08:00
Jeremy Fitzhardinge
408e7665d9 mononoke: move RepositoryId into mononoke-types
Summary: There's nothing Mercurial-specific about identifying a repo. This also outright removes some dependencies on mercurial-types.

Reviewed By: StanislavGlebik

Differential Revision: D13512616

fbshipit-source-id: 4496a93a8d4e56cd6ca319dfd8effc71e694ff3e
2018-12-19 10:24:27 -08:00
Liubov Dmitrieva
473a9ba741 add api for using PhaseHint and Db without memcache
Summary:
New HintPhases api that will be used for tests vs CachingHintPhases that will be used in prod.

The api will be owned by RepoClient.

We will check what type of BlobRepo we have (i.e. blob:rocks, blob:files, blob:remote etc) and build the phases depending on the type of the BlobRepo.

We will use either CachingHintPhases backed by real db / MyRouter or HintPhases backed by Sqlite.

Reviewed By: StanislavGlebik

Differential Revision: D13466225

fbshipit-source-id: 06ea565171d8ea8d7335fbbd91d86fbdcc01c8fc
2018-12-14 09:20:29 -08:00
Liubov Dmitrieva
776c48ca40 add logic for slow path phases calculation
Summary: Calculation is based on beeing ancestor of a public bookmark

Reviewed By: StanislavGlebik

Differential Revision: D13441622

fbshipit-source-id: e20df656847913bc124b491aaeb2660d21c85da1
2018-12-14 08:06:53 -08:00
Liubov Dmitrieva
120849c0be memcache getter/setter for phases
Summary:
This diff includes the logic on how we will receive a phase for a given commit based on memcahe / db lookup and refresh, and slow path calculation.

It has a blank place of slow path (not found in the memcache, not found in the db => we have to calculate the phase based on being ancestor of public bookmark).

Collecting the stats should be added separately.

Reviewed By: StanislavGlebik

Differential Revision: D13415481

fbshipit-source-id: 6a4cb5b8dfbb0d7b2535d903c653bbf7a088c422
2018-12-14 08:06:53 -08:00
Liubov Dmitrieva
ba58aa252a phases support: mysql add/get
Summary: add/get api for adding phases to mysql phases table

Reviewed By: lukaspiatkowski

Differential Revision: D13376701

fbshipit-source-id: b71e52db2c30b59b0070f49327bfdd189c28d6cc
2018-12-10 06:14:03 -08:00