mirror of
https://github.com/facebook/sapling.git
synced 2024-10-16 11:52:02 +03:00
640f272598
Summary: Migrate the configuration of sql data managers from the old configuration using `sql_ext::SqlConstructors` to the new configuration using `sql_construct::SqlConstruct`. In the old configuration, sharded filenodes were included in the configuration of remote databases, even when that made no sense: ``` [storage.db.remote] db_address = "main_database" sharded_filenodes = { shard_map = "sharded_database", shard_num = 100 } [storage.blobstore.multiplexed] queue_db = { remote = { db_address = "queue_database", sharded_filenodes = { shard_map = "valid_config_but_meaningless", shard_num = 100 } } ``` This change separates out: * **DatabaseConfig**, which describes a single local or remote connection to a database, used in configuration like the queue database. * **MetadataDatabaseConfig**, which describes the multiple databases used for repo metadata. **MetadataDatabaseConfig** is either: * **Local**, which is a local sqlite database, the same as for **DatabaseConfig**; or * **Remote**, which contains: * `primary`, the database used for main metadata. * `filenodes`, the database used for filenodes, which may be sharded or unsharded. More fields can be added to **RemoteMetadataDatabaseConfig** when we want to add new databases. New configuration looks like: ``` [storage.metadata.remote] primary = { db_address = "main_database" } filenodes = { sharded = { shard_map = "sharded_database", shard_num = 100 } } [storage.blobstore.multiplexed] queue_db = { remote = { db_address = "queue_database" } } ``` The `sql_construct` crate facilitates this by providing the following traits: * **SqlConstruct** defines the basic rules for construction, and allows construction based on a local sqlite database. * **SqlShardedConstruct** defines the basic rules for construction based on sharded databases. * **FbSqlConstruct** and **FbShardedSqlConstruct** allow construction based on unsharded and sharded remote databases on Facebook infra. * **SqlConstructFromDatabaseConfig** allows construction based on the database defined in **DatabaseConfig**. * **SqlConstructFromMetadataDatabaseConfig** allows construction based on the appropriate database defined in **MetadataDatabaseConfig**. * **SqlShardableConstructFromMetadataDatabaseConfig** allows construction based on the appropriate shardable databases defined in **MetadataDatabaseConfig**. Sql database managers should implement: * **SqlConstruct** in order to define how to construct an unsharded instance from a single set of `SqlConnections`. * **SqlShardedConstruct**, if they are shardable, in order to define how to construct a sharded instance. * If the database is part of the repository metadata database config, either of: * **SqlConstructFromMetadataDatabaseConfig** if they are not shardable. By default they will use the primary metadata database, but this can be overridden by implementing `remote_database_config`. * **SqlShardableConstructFromMetadataDatabaseConfig** if they are shardable. They must implement `remote_database_config` to specify where to get the sharded or unsharded configuration from. Reviewed By: StanislavGlebik Differential Revision: D20734883 fbshipit-source-id: bb2f4cb3806edad2bbd54a47558a164e3190c5d1 |
||
---|---|---|
.. | ||
main.rs |