Commit Graph

22 Commits

Author SHA1 Message Date
Lukas Piatkowski
515a2909eb monononoke hashes: remove usages of borrows of hashes which are Copy
Summary: The Copy trait means that something is so cheap to copy that you don't even need to explicitly do `.clone()` on it. As it doesn't make much sense to pass &i64 it also doesn't make much sense to pass &<Something that is Copy>, so I have removed all the occurences of passing one of ouf hashes that are Copy.

Reviewed By: fanzeyi

Differential Revision: D13974622

fbshipit-source-id: 89efc1c1e29269cc2e77dcb124964265c344f519
2019-02-06 15:11:35 -08:00
Jeremy Fitzhardinge
408e7665d9 mononoke: move RepositoryId into mononoke-types
Summary: There's nothing Mercurial-specific about identifying a repo. This also outright removes some dependencies on mercurial-types.

Reviewed By: StanislavGlebik

Differential Revision: D13512616

fbshipit-source-id: 4496a93a8d4e56cd6ca319dfd8effc71e694ff3e
2018-12-19 10:24:27 -08:00
Lukas Piatkowski
d8476a6527 mononoke: pass CoreContext down to filenodes
Reviewed By: StanislavGlebik

Differential Revision: D13304158

fbshipit-source-id: d73597ed64822d7a986f6a8d46a96cfbbe99a8e7
2018-12-04 01:16:32 -08:00
Jeremy Fitzhardinge
ec86d453ae rust/stats: don't require users to import lazy_static
Summary:
Now that Rust macros can be `use`d like normal symbols, `stats` can
simply import the `lazy_static!` macro without requiring its users to do it.

Reviewed By: Imxset21

Differential Revision: D13281897

fbshipit-source-id: a6780fbace07dd784308e642d4a384322a17c367
2018-12-02 14:34:39 -08:00
Stanislau Hlebik
b058eabc13 mononoke: memcache for changesets
Reviewed By: jsgf

Differential Revision: D10505048

fbshipit-source-id: 1a7accbce29b952a5c8dc99a985eb528d282ddef
2018-10-25 02:08:53 -07:00
Stanislau Hlebik
7b9d0fb5df mononoke: memcache for filenodes
Summary:
We have a problem with service upgrades/restarts because many servers start
sending too many requests to mysql db.

Let's add a memcache that will prevent that.

Reviewed By: jsgf

Differential Revision: D10488624

fbshipit-source-id: 4575d359bc269e29fe72b47d7f47cda22bf4acd7
2018-10-25 02:08:53 -07:00
Stanislau Hlebik
5dfb4d76b8 common/rust: renamed get_cached to get_cached_or_fill
Summary:
In the next diff I'm going to add a separate functions to look in the cache and
insert into the cache, so rename it to avoid confusion

Reviewed By: Imxset21

Differential Revision: D9869648

fbshipit-source-id: f2bd806b14d78660518d841d90a903970028eb37
2018-09-18 02:06:43 -07:00
Simon Farnsworth
3733848cef Use Abomonation and cachelib to replace Asyncmemo
Summary:
Asyncmemo has two issues for our use:

1. Separate memory pool from cachelib caches.
2. Future fusion means that a `get` that should succeed will fail because there
was an earlier get still in progress.

The second is good for memoization, where the worst case from a failed get is
extra CPU work, but not so good for caching. Replace uses of Asyncmemo for
caches with a cachelib based cache

Reviewed By: StanislavGlebik

Differential Revision: D9013679

fbshipit-source-id: b85d4eec7294e0c8ee08faa671d26901b35cf1fc
2018-08-07 13:37:09 -07:00
Simon Farnsworth
1ffa07da46 Derive Abomonation for all interesting types
Summary:
These are the types that we currently need to be able to serialize if we're to
replace `Asyncmemo`'s caching uses with cachelib. Derive the `Abomonation`
trait for all of them.

Reviewed By: jsgf

Differential Revision: D9082597

fbshipit-source-id: 910e90476a3cc4d18ba758226b8572c3e8d264c6
2018-08-07 13:37:08 -07:00
Lukas Piatkowski
8748c72eea filenodes: use Memcache to cache get_all_filenodes results
Summary: Because get_all_filenodes can go above 1MB (but still less than 3MB) I added a chunking logic to speed up fetching of even the long histories

Reviewed By: farnz

Differential Revision: D9029774

fbshipit-source-id: c2d46a4b49bfebd3c778a0b9c5cf226857f34768
2018-07-27 13:21:20 -07:00
Lukas Piatkowski
4cf8e881e8 filenodes: move blake2_path_hash to a common place for all filenodes implementations
Summary: The blake2_path_hash is used only in MySql right now, but I want to use it in caching as well, so moving it to a more accessible place.

Reviewed By: jsgf

Differential Revision: D9028799

fbshipit-source-id: b055550fbfca5c6c419702490504f08812592470
2018-07-27 13:21:20 -07:00
Lukas Piatkowski
53b147435c filenodes: track the size of get_all_filenodes requests in thrift compact form
Summary:
This information is useful to decide if get_all_filenodes is small enough to fit into memcache.
Later it will be used to track how many elements are fitting into memcache nicely.

Reviewed By: jsgf

Differential Revision: D9028395

fbshipit-source-id: ae71486f9e9d60d7b54ce28b8c591c09b8925947
2018-07-27 13:21:20 -07:00
Lukas Piatkowski
cf906c3056 filenodes: move CachingFilenodes to it's own module
Summary: a bit of cleanup for the upcoming diffs

Reviewed By: farnz

Differential Revision: D9014639

fbshipit-source-id: d03e4969426eced70fd19b91a2c15b607bf52b39
2018-07-27 08:21:14 -07:00
Lukas Piatkowski
6b934172a6 filenodes: add thrift structures for FilenodeInfo
Summary: Those structures will be used in next diffs to store the FilenodeInfo inside memcache for caching purposes

Reviewed By: farnz

Differential Revision: D9014213

fbshipit-source-id: 4952a90415d4b8ab903387fd5cdfaf08d9870c07
2018-07-27 08:21:13 -07:00
Lukas Piatkowski
985e370f87 asyncmemo: add stats to track size of asyncmemo cache
Reviewed By: jsgf

Differential Revision: D8677107

fbshipit-source-id: f3ef30f36c918c1d4e06d3e2e7856ce429bad6cf
2018-06-28 09:21:31 -07:00
Lukas Piatkowski
4f44c3f130 mercurial_types: remove D* types and move mercurial types around
Summary:
Now it is as it should be: mercurial_types have the types, mercurial has revlog related structures
burnbridge

Reviewed By: farnz

Differential Revision: D8319906

fbshipit-source-id: 256e73cdd1b1a304c957b812b227abfc142fd725
2018-06-07 13:19:16 -07:00
Stanislau Hlebik
0b93cff8d4 mononoke: add a method to fetch all filenodes at once
Summary:
It will be used in getfiles wireproto method. It requires fetching the whole
file history, and doing lots of point lookups is too slow.

Reviewed By: lukaspiatkowski

Differential Revision: D8182050

fbshipit-source-id: b17cb2259b2237f0027bfb499fbe97d9b40b1a62
2018-05-29 04:11:47 -07:00
Stanislau Hlebik
b3432410ae mononoke: use caching for filenodes
Summary: As with changesets and blobs, let's cache filenodes data

Reviewed By: jsgf

Differential Revision: D7831105

fbshipit-source-id: 334cb474f5cc3ef8dba0945d11273b2b3875e8ad
2018-05-03 10:31:32 -07:00
Lukas Piatkowski
73034a99b2 CODEMOD: rename mercurial_types::HgFileNodeId to DFileNodeId
Summary: mercurial_types::DFileNodeId should be replaced by types from mononoke_types in most cases. This rename should help with tracking this

Reviewed By: sid0

Differential Revision: D7619290

fbshipit-source-id: aa6a8e55ae3810c4531028c3b3db2e5730fe7846
2018-04-16 03:40:24 -07:00
Lukas Piatkowski
f6c3f72745 CODEMOD: rename mercurial_types::HgChangesetId to DChangesetId
Summary: mercurial_types::DChangesetId should be replaced by types from mononoke_types in most cases and by mercurial::HgChangesetId in others. This rename should help with tracking this

Reviewed By: sid0

Differential Revision: D7618897

fbshipit-source-id: 78904f57376606be99b56662164e0c110e632c64
2018-04-16 03:40:24 -07:00
Stanislau Hlebik
27e4aab5b4 add Send + Sync to Filenodes
Summary: It will be used in BlobRepo, and it requires it to be Send + Sync

Reviewed By: farnz

Differential Revision: D7429058

fbshipit-source-id: 610051ba3ed7ce66dd8058e1ba9fabb3ffd12c4e
2018-03-29 03:29:57 -07:00
Stanislau Hlebik
f550dfc41d mononoke: diesel filenodes implementation
Summary:
Initial support for filenodes using diesel.
These filenodes implementation is very close to what hg stores in revlog files.
Currently this implementation doesn't support copies.
This issue will be fixed in the next diffs.

Reviewed By: farnz

Differential Revision: D7305910

fbshipit-source-id: bbb0196dbee24c3a99d58055de11d433a518e40b
2018-03-28 03:25:27 -07:00