Summary:
It makes startup unbearably slow, and doesn't add any benefits at all. Revert
it
Reviewed By: purplefox
Differential Revision: D9358741
fbshipit-source-id: 26469941304f737c856a6ffca5e577848ad30955
Summary:
Should be functionally equivalent and semantically more appropriate
This also makes a couple of small API changes:
- The inner function is expected to just return a Result - IntoFuture is
overkill if its supposed to be synchronous in the first place
- `asynchronize` itself returns `impl Future` rather than being intrinsically
boxed.
- Restructure dieselfilenodes::add_filenodes to only asynchronize the insert
itself.
Reviewed By: farnz
Differential Revision: D8959535
fbshipit-source-id: fef9164e3be0069bd0d93573642cd57bb5babb73
Summary: The blake2_path_hash is used only in MySql right now, but I want to use it in caching as well, so moving it to a more accessible place.
Reviewed By: jsgf
Differential Revision: D9028799
fbshipit-source-id: b055550fbfca5c6c419702490504f08812592470
Summary:
Unify all uses of Sqlite and of Mysql
This superceded D8712926
Reviewed By: farnz
Differential Revision: D8732579
fbshipit-source-id: a02cd04055a915e5f97b540d6d98e2ff2d707875
Summary: we want most of the read traffic to be handled by replicas
Reviewed By: lukaspiatkowski
Differential Revision: D8548027
fbshipit-source-id: 297b070b5b33ec114026ecf3f1079fdb26f54b19
Summary: There shouldn't be more than one thread writing to the database, because it causes lags in slaves and they race for database locks between themselves. One write connection should be sufficient enough.
Reviewed By: StanislavGlebik
Differential Revision: D8348604
fbshipit-source-id: ceef081ed89611978accfa55969883078d65a58f
Summary: this will make it easier to identify stats exported from mononoke vs external libraries
Reviewed By: StanislavGlebik
Differential Revision: D8331418
fbshipit-source-id: c151e76aa386fb13759fced7cc07b03ac67fe051
Summary: Going to take a while to get to everything, but here's a start.
Reviewed By: Imxset21
Differential Revision: D8311107
fbshipit-source-id: ada1908b320a5277eda2587d7e8f26b13b952154
Summary:
Now it is as it should be: mercurial_types have the types, mercurial has revlog related structures
burnbridge
Reviewed By: farnz
Differential Revision: D8319906
fbshipit-source-id: 256e73cdd1b1a304c957b812b227abfc142fd725
Summary:
The new_blobimport job is having difficulties when the pool is too large, because the write transactions are taking too long. If the pool is configured to be 1 for it then everything seems fine and fast enough.
On the other hand the Mononoke server should have bigger connectino pool size to be able to quickly respond for read requests.
Reviewed By: farnz
Differential Revision: D8235413
fbshipit-source-id: 84e0013ce569c3f103a2096001605aab828d178c
Summary:
The transaction is not necessary in there, since if any of the query fails then the commit is not written in changesets table.
We need to remove it, because with many writes to the filenodes table some of the transactions are timing out while waiting for the database's lock for transation
Reviewed By: farnz
Differential Revision: D8220722
fbshipit-source-id: 003b1369abb1a47e8ad37517745ab659116b95f6
Summary: It only needs to borrow them.
Reviewed By: kulshrax
Differential Revision: D8244267
fbshipit-source-id: 2a24a3b7c6eb65177e4e26c57650dd7e096b4202
Summary:
It will be used in getfiles wireproto method. It requires fetching the whole
file history, and doing lots of point lookups is too slow.
Reviewed By: lukaspiatkowski
Differential Revision: D8182050
fbshipit-source-id: b17cb2259b2237f0027bfb499fbe97d9b40b1a62
Summary:
Move functionality of converting FilenodeRow into FilenodeInfo into a separate
function. It will be used in the next diff
Reviewed By: lukaspiatkowski
Differential Revision: D8182051
fbshipit-source-id: f2276aa979200f71aa05808595842c09a9f5899f
Summary:
Motivation for this change is the following:
We want to avoid doing point lookups to the db to fetch the history, and
instead select the history for a file or a directory with a single select. That
should be many times faster.
After we select the data from filenodes table, we need to fetch copy info
data. To do that we need to either do lots of point lookups to the fixedcopyinfo (which
undermines the point of doing single select) or construct a giant select
statement. First option is inefficient, the second can introduce big code
complexity.
Instead let's add a single field that says whether we have fixed copy info or
not. Db size increase should be tiny.
For the data that we've already imported I'm planning to run a script that will
fill the has_copyinfo field correctly.
Reviewed By: jsgf
Differential Revision: D8164029
fbshipit-source-id: c91c99b065808a93a9b361914cf9b3822d78cb60
Summary: It can be useful to understand how often do we hit a db, write to blobstore, etc
Reviewed By: farnz
Differential Revision: D7806954
fbshipit-source-id: 9c85cbb11e184ae9f7751e05e961980b29b7568a
Summary:
Update x86 and aarch64 toolchains, rust-crates-io update.
The `use_nested_groups` feature is now stable, so we don't need to list it in
`#![feature(...)]` lines.
Reviewed By: kulshrax
Differential Revision: D7690730
fbshipit-source-id: 5f74bb01df6bd88a66ac00a82e446c18e3a1c43d
Summary:
Just as with changesets, let's make filenodes async.
I had to make unit-tests use async_unit, otherwise tokio::spawn() failed with
SpawnedError. I'm not sure why it happened, but async_unit has fixed the issue
Reviewed By: lukaspiatkowski
Differential Revision: D7635554
fbshipit-source-id: c6130b95cee2d60d0b6eec0ab6c1b3e4f3a26ccd
Summary:
Add functions that return a db connection.
This is the first step to make filenodes truly async.
The code is very similar to Diesel Changesets implementation
Reviewed By: lukaspiatkowski
Differential Revision: D7585297
fbshipit-source-id: 93df9ccbbdf0c29d6c977fec83444bc5ba9aa707
Summary: mercurial_types::DFileNodeId should be replaced by types from mononoke_types in most cases. This rename should help with tracking this
Reviewed By: sid0
Differential Revision: D7619290
fbshipit-source-id: aa6a8e55ae3810c4531028c3b3db2e5730fe7846
Summary: mercurial_types::DChangesetId should be replaced by types from mononoke_types in most cases and by mercurial::HgChangesetId in others. This rename should help with tracking this
Reviewed By: sid0
Differential Revision: D7618897
fbshipit-source-id: 78904f57376606be99b56662164e0c110e632c64
Summary: mercurial_types::NodeHash should be replaced by types from mononoke_types in most cases and by mercurial::NodeHash in others. This rename should help with tracking this fact.
Reviewed By: sid0
Differential Revision: D7618389
fbshipit-source-id: a876e723d911df626c7851fba56a056843b4e049
Summary:
open_or_create is useful for testing.
insert_chunk_size sets the mysql batch size
Reviewed By: farnz
Differential Revision: D7429043
fbshipit-source-id: 9d651a2f71abb9022c5390d059e4298efb49f38c
Summary: To query all the parents of the filenode. This emulates getfiles request
Reviewed By: farnz
Differential Revision: D7427133
fbshipit-source-id: 54d054fc7a30db027aa327d3be3b93c86dd3a223
Summary:
Previously all the copy information was ignored. In this diff we are using it.
Note that `paths` table is also used now - it is used to convert from hash of
the path to the real path.
Reviewed By: farnz
Differential Revision: D7381437
fbshipit-source-id: 13f8bf0f269cf40d06693dd0cdca15da40c1ab64
Summary:
Initial support for filenodes using diesel.
These filenodes implementation is very close to what hg stores in revlog files.
Currently this implementation doesn't support copies.
This issue will be fixed in the next diffs.
Reviewed By: farnz
Differential Revision: D7305910
fbshipit-source-id: bbb0196dbee24c3a99d58055de11d433a518e40b
