Summary:
As a starting point for tuning up caching, ask for huge evenly sized
pools. We will gather data from here via Cachelib's logging, and then come back
to tune this properly
Reviewed By: StanislavGlebik
Differential Revision: D9323841
fbshipit-source-id: 721f24acaca4542276b73d25985e54b111e2ca8a
Summary:
Now that we can configure access configs, ask for one suited for a
larger cache
Reviewed By: StanislavGlebik
Differential Revision: D9323842
fbshipit-source-id: 40d94090121214ed3310b40ce8de8a7c350ed4af
Summary:
The process size shrinker has issues in Sandcastle CI; the Tupperware
shrinker will only enable shrinking if we're inside an identifiable Tupperware
task.
Switch to the Tupperware shrinker, which also means that we're using
`cachelib`'s default recommendations for shrinking the cache under memory
pressure in Tupperware.
Reviewed By: lukaspiatkowski
Differential Revision: D9297077
fbshipit-source-id: 0ec788e32409e2a5d058a89e11dc1f7e37c7552f
Summary: - Make `Bookmakrs` work with `ChangsetId` instead of `HgChangesetId`
Reviewed By: StanislavGlebik, farnz
Differential Revision: D9297139
fbshipit-source-id: e18661793d144669354e509271044410caa3502a
Summary: There's a hidden dependency here -- ideally it'd be made explicit.
Reviewed By: jsgf
Differential Revision: D9307939
fbshipit-source-id: e00a201e20bb8d83be09eab10cfdd2ce03a3e22f
Summary:
Set a panichandler by default in cmdlib::get_logger to make sure
everyone gets one set. It configures itself to exit the process so that we
don't leave it in a half-broken state.
The Mononoke server was already using a panic hook, but this replaces it with
one that prints more detail about what was going on at the time.
Reviewed By: StanislavGlebik
Differential Revision: D9234587
fbshipit-source-id: bb51790a60b1ee545a364b4b92e09ec950788684
Summary:
Simply make the right init call, so that we have data in Scuba that
the cachelib team can use to see if we're doing the right thing
Reviewed By: StanislavGlebik
Differential Revision: D9195015
fbshipit-source-id: ff31dd9462a489bc5f8070068effd8a1bfb2fa58
Summary:
Asyncmemo has two issues for our use:
1. Separate memory pool from cachelib caches.
2. Future fusion means that a `get` that should succeed will fail because there
was an earlier get still in progress.
The second is good for memoization, where the worst case from a failed get is
extra CPU work, but not so good for caching. Replace uses of Asyncmemo for
caches with a cachelib based cache
Reviewed By: StanislavGlebik
Differential Revision: D9013679
fbshipit-source-id: b85d4eec7294e0c8ee08faa671d26901b35cf1fc
Summary:
The pool rebalancer allows cachelib to shed mostly-dead slabs in
favour of new allocations. Turn it on.
Reviewed By: jsgf
Differential Revision: D9035995
fbshipit-source-id: 1be8073381020b4119df5ffba1a16e90ab7233be
Summary:
We'll be running in Tupperware, and want to shrink when we get too
large to avoid OOM due to caches. Configure cachelib appropriately
Reviewed By: StanislavGlebik
Differential Revision: D8900371
fbshipit-source-id: 4f1f64c2508c64e4ce2d201e0a0e86446f84ffef
Summary:
Now bonsai changesets are created at the same time as hg changesets, and
the mapping between bonsai and hg changesets is recorded
One important piece is missing. At the moment copy info information is ignored.
I'll add it in the next diffs.
Before diff is landed, I re-run the blobimport to prefill missing bonsai changesets.
Reviewed By: farnz
Differential Revision: D8893504
fbshipit-source-id: 1cc4bbcca2f489a0ef6990d6c04d5b3fd8bef92c
Summary:
Back out "[mononoke] Switch to cachelib for blob caching"
Original commit changeset: 2549d85dfcba
Back out "[mononoke] Remove unused asyncmemo imports"
Original commit changeset: e34f8c34a3f6
Back out "mononoke: fix blobimport"
Original commit changeset: b540201b93f1
Reviewed By: StanislavGlebik
Differential Revision: D8989404
fbshipit-source-id: e4e7c629cb4dcf196aa56eb07a53a45f6008eb4e
Summary: Makes it easier to understand the progress
Reviewed By: farnz
Differential Revision: D8989178
fbshipit-source-id: dcfebc12cf70871b3353a2cab456486c74829dd8
Summary:
These types are Hg specific. Since we are going to add bonsai changeset
creation soon, let's make it clear in the types
Reviewed By: farnz
Differential Revision: D8911359
fbshipit-source-id: 8b6cc45122402d7b7e074e66d904d979030de705
Summary:
cachelib can shrink caches to avoid running into OOM, while still
giving as close to full-sized cache performance as possible. Make it possible
for Rust users of cachelib to request cache shrinking on low memory, while
choosing a strategy that suits them.
Reviewed By: Imxset21
Differential Revision: D8895814
fbshipit-source-id: ca4eb5b002c9ed922e7f7f56de002313a0d2303b
Summary:
Start deprecating AsyncMemo for caching purposes, by removing its use
as a blobstore cache.
Reviewed By: StanislavGlebik
Differential Revision: D8840496
fbshipit-source-id: 2549d85dfcba6647e9b0824ab55ab76165a17564
Summary:
I don't like glog for interactive use at all. But keep it as the
default for blobimport, and add a flag to change it.
Reviewed By: StanislavGlebik
Differential Revision: D8909674
fbshipit-source-id: d0b9c439f72f231c95e9109e16b30e87cfaa2eed
Summary: make it easier to import mononoke-config repo, can be used in development, but it also will be used in fbpkg in further diffs
Reviewed By: farnz
Differential Revision: D8887356
fbshipit-source-id: 6dbcccff7fedc6a72be1b8240e49d4fba8cf89a1
Summary: later this library will be used by admin tool to ease importing of config repo
Reviewed By: StanislavGlebik
Differential Revision: D8882178
fbshipit-source-id: 293a26b038f8d76e9fcedb72a4041a48f502a00a
Summary: This will be useful for the bonsai verification tool.
Reviewed By: StanislavGlebik
Differential Revision: D8792562
fbshipit-source-id: f409d0fa042528b04462a1539fd3c2a8064a4f6e
Summary: This code can easily be shared.
Reviewed By: StanislavGlebik
Differential Revision: D8777307
fbshipit-source-id: f11314f6a63bb191dc38d07cec181a4b05b158d9
Summary:
I want to use this code for the bonsai verification tool without
copying it around. It's also a nice simplification of the code.
Reviewed By: jsgf
Differential Revision: D8675825
fbshipit-source-id: cf9432069f154dafeb81a5e93a2fb5cbd717b981
