Summary:
This adds changes, introduced by me in D21841293.
For my stuff: I will only land this once the configerator diff is reviewed.
Reviewed By: farnz
Differential Revision: D21862276
fbshipit-source-id: d4f5ccdd4ef9d62bb1e394d4db36930413633fa6
Summary:
This diff adds support for the `version_name` field, coming from the
`commitsyncmap` config, stored in the configerator.
Note: ATM, this field is optional in the thrift config, but once we get past
the initial deployment stage, I expect it to be present always. This is why
in `CommmitSyncConfig` I make it `String` (with a default value of `""`) rather
than `Option<String>`. The code, which will be writing this value into
`synced_commit_mapping` should not ever care whether it's present or not, since
every mapping should always have a `version_name` present.
Reviewed By: StanislavGlebik
Differential Revision: D21764641
fbshipit-source-id: 35a7f487acf0562b309fd6d1b6473e3b8024722d
Summary:
The blobstore multiplexer contains logic to log blobstore operations to
scuba, as well as updating `PerfCounters`. There are some cases where we don't use the
multiplexed blobstore, which means that we're missing this important logging.
Factor out the logging to a separate crate and implement `LogBlob`, which wraps
another blobstore and performs logging to both scuba and PerfCounters.
Reviewed By: StanislavGlebik
Differential Revision: D21573455
fbshipit-source-id: 490ffd347f1ad19effc93b93f880836467b87651
Summary:
The configerator thrift file was updated to remove a path from hook
config. However, this change wasn't synced to fbsource.
Sync the change and fix up the tests that are broken by this change.
Reviewed By: krallin
Differential Revision: D21594221
fbshipit-source-id: 7b64180914f6c6802e4d70fcb1a5d6ec36eb2eac
Summary: A result of `configerator-thrift-updater scm/mononoke/repos/repos.thrift`
Reviewed By: farnz
Differential Revision: D21350982
fbshipit-source-id: b3344c99c6f53c727ea16ebc0f81f90527de103d
Summary:
Currently, Mononoke's configs are loaded at startup and only refreshed
during restart. There are some exceptions to this, including throttling limits.
Other Mononoke services (such as the LFS server) have their own implementations
of hot reloadable configs, however there isn't a universally agreed upon method.
Static configs makes it hard to roll out features gradually and safely. If a
bad config option is enabled, it can't be rectified until the entire tier is
restarted. However, Mononoke's code is structured with static configs in mind
and doesn't support hot reloading. Changing this would require a lot of work
(imagine trying to swap out blobstore configs during run time) and wouldn't
necessarily provide much benefit.
Instead, add a subset of hot reloadable configs called tunables. Tunables are
accessible from anywhere in the code and are cheap to read as they only require
reading an atomic value. This means that they can be used even in hot code
paths.
Currently tunables support reloading boolean values and i64s. In the future,
I'd like to expand tunables to include more functionality, such as a rollout
percentage.
The `--tunables-config` flag points to a configerator spec that exports a
Tunables thrift struct. This allows differents tiers and Mononoke services to
have their own tunables. If this isn't provided, `MononokeTunables::default()`
will be used.
This diff adds a proc_macro that will generate the relevant `get` and `update`
methods for the fields added to a struct which derives `Tunables`. This struct is
then stored in a `once_cell` and can be accessed using `tunables::tunables()`.
To add a new tunable, add a field to the `MononokeTunables` struct that is of
type `AtomicBool` or `AtomicI64`. Update the relevant tunables configerator
config to include your new field, with the exact same name.
Removing a tunable from `MononokeTunables` is fine, as is removing a tunable
from configerator.
If the `--tunables-config` path isn't passed, then a default tunables config
located at `scm/mononoke/tunables/default` will be loaded. There is also the
`--disable-tunables` flag that won't load anything from configerator, it
will instead use the `Tunable` struct's `default()` method to initialise it.
This is useful in integration tests.
Reviewed By: StanislavGlebik
Differential Revision: D21177252
fbshipit-source-id: 02a93c1ceee99066019b23d81ea308e4c565d371
Summary: The configerator structs are used in many top level functions in Mononoke and are required in order to build all the code on github
Reviewed By: ahornby
Differential Revision: D21130546
fbshipit-source-id: 7f17d92173f5ecf7c3406ae4202359a0db8df84a
Summary: I'm going to want to be able to test against a single ephemeral shard, as well as production use against a real DB. Use the standard config to make that possible.
Reviewed By: ahornby
Differential Revision: D21048697
fbshipit-source-id: 644854e2c831a9410c782ca1fddc1c4b5f324d03
Summary:
See D20941946 for why this is being added. This just brings in the updated
Thrift definition.
Reviewed By: farnz
Differential Revision: D20942176
fbshipit-source-id: c060f80666cb79f1498023276b7a09ec12bf52b4
Summary:
This diff turns off the support_old_nightly feature of async-trait (https://github.com/dtolnay/async-trait/blob/0.1.24/Cargo.toml#L28-L32) everywhere in fbcode. I am getting ready to remove the feature upstream. It was an alternative implementation of async-trait that produces worse error messages but supports some older toolchains dating back to before stabilization of async/await that the default implementation does not support.
This diff includes updating async-trait from 0.1.24 to 0.1.29 to pull in fixes for some patterns that used to work in the support_old_nightly implementation but not the default implementation.
Differential Revision: D20805832
fbshipit-source-id: cd34ce55b419b5408f4f7efb4377c777209e4a6d
Summary: This diff makes it possible to relay on the thrift structures from configerator in OSS.
Reviewed By: ahornby
Differential Revision: D20279457
fbshipit-source-id: 39df1c7a6f78b10a0a5bdeeebe476249ab674cc8
Summary:
This is convenient because it makes it possible to tell what is going on within
Mononoke from the outside (e.g. introspect perf counters).
I'll land this after D20387115.
Reviewed By: farnz
Differential Revision: D20387125
fbshipit-source-id: ee070c4d658a0ec8f232152fe8b34bd0b56e6888
Summary:
This incorporates microwave into the cache warmup process. See earlier in this
stack for a description of what this does, how it works, and why it's useful.
Reviewed By: ahornby
Differential Revision: D20219904
fbshipit-source-id: 52db74dc83635c5673ffe97cd5ff3e06faba7621
