Summary: It can be useful to understand how often do we hit a db, write to blobstore, etc
Reviewed By: farnz
Differential Revision: D7806954
fbshipit-source-id: 9c85cbb11e184ae9f7751e05e961980b29b7568a
Summary:
If baserev == Some(idx), we changed it to None. Also we can have None if
baserev == -1 in mercurial. However these two cases are different. In the first
case it means that we have a literal chunk, not delta. In the second case it
means that we have a delta against empty string! So this is technically almost
the same, except that delta against empty string also have a 12 bytes prefix.
Previously this prefix was used as part of a revision data.
This diff fixes it.
Reviewed By: quark-zju
Differential Revision: D7815713
fbshipit-source-id: def2e54b2cc7379ba8f931ecf3f3c0c38d716058
Summary:
We want to be able to identify "interesting" sessions - add a Scuba
sample that tells us what wireprotocol commands were sent, and how long the
session lasted.
Reviewed By: jsgf
Differential Revision: D7813906
fbshipit-source-id: a9bd48996a60b41098243f6c815465cd33d1429c
Summary:
We want to be able to tie up our logs with a session identifier, so
that we can see if a given slow operation is expected (a complex session) or a
surprise. Give the client a logger that has a session ID tied in.
Reviewed By: jsgf
Differential Revision: D7813908
fbshipit-source-id: bc8a59046c19a0939fd044ca8b5b7ccc2956c5f0
Summary:
These have bitrotted, but I need a test case for a Scuba change. Fix
them.
Reviewed By: StanislavGlebik
Differential Revision: D7813907
fbshipit-source-id: e4e9b01a8a3c1de27f59c6d5ea695152df99d4ff
Summary: Useful utility that let's you f.e. fetch blob of data from manifold, decode it and show it to you
Reviewed By: jsgf
Differential Revision: D7779154
fbshipit-source-id: aaa4ae1d09b64f7f52c7942a51e8bb4ccc0cb700
Summary: The deltas are meant to be applied on raw filenode content that has incorporated copy/move information. Make it so
Reviewed By: jsgf
Differential Revision: D7777329
fbshipit-source-id: 23ce90269cdf5dbb8a16aab026c031c2af68fccb
Summary: The current behavior of delta::apply will panic Mononke server when the client sends malformed delta request. This change will instead propagate an Error explaining why the delta was malformed.
Reviewed By: jsgf
Differential Revision: D7775544
fbshipit-source-id: a64c27c7b9f13323b8be70eb8c2cdf315ce8f08d
Summary:
We'd like to move away from `RawNodeBlob` and `RawCSBlob` to data structures
serialized by Thrift. This is the first step to doing that.
The most important thing here is that it reuses file content IDs from native
Mononoke storage.
Reviewed By: jsgf
Differential Revision: D7771990
fbshipit-source-id: de4ee0b56aa6610caeff84b2235e19855df086cb
Summary: Want to get this in now before I forget.
Reviewed By: jsgf
Differential Revision: D7781551
fbshipit-source-id: a5e6fa062514d90113c96307f78e20fa05f770ea
Summary: Will be used for Thrift envelopes.
Reviewed By: jsgf
Differential Revision: D7771214
fbshipit-source-id: 3f0dbf77793064f2606ebe34672629c4e49cc7fe
Summary: Want to use this in future work
Reviewed By: jsgf
Differential Revision: D7771139
fbshipit-source-id: 31115f9bd119f23480572f69f56e956eeb43a6c6
Summary: This is an important consistency check for bonsai changesets.
Reviewed By: jsgf
Differential Revision: D7755664
fbshipit-source-id: 4e64ed532d0730147efec7a2b10b61f625e50dd0
Summary:
This does mean that we need to adjust QuickCheck generation a bit to ensure
validity.
Reviewed By: jsgf
Differential Revision: D7755635
fbshipit-source-id: ebca6f5a52038b64f83ff599079460daf061cb44
Summary: the idea of Mercurial heads in Mononoke will be represented by bookmarks, so there is no need to have them around
Reviewed By: StanislavGlebik
Differential Revision: D7775032
fbshipit-source-id: 1618a1e51862d7c115b2955082f40ee890a045f1
Summary: I got annoyed at the lack of this.
Reviewed By: StanislavGlebik
Differential Revision: D7755636
fbshipit-source-id: 0d519dca94ed93e3efe8d12f9004661b92fb8b1a
Summary: Just as with other dirs, let's fetch top-level dirs in parallel.
Reviewed By: jsgf
Differential Revision: D7724901
fbshipit-source-id: 55a113d64201253b357ed37017ce0596647327a9
Summary: Although this change is breaking with the traditional definition of heads in Mercurial, we decided that using bookmarks as heads is sufficient for fbsource use case as well as it matches the git model where every unnamed branch is garbage collected.
Reviewed By: jsgf
Differential Revision: D7745541
fbshipit-source-id: 2ec716db7799bf7bab8b2b77a85d1521a880f3df
Summary:
This redefines file changes and deletes slightly -- see the comments
for a full description.
Reviewed By: jsgf, farnz
Differential Revision: D7750040
fbshipit-source-id: f1c52295cd21150fbdc909198104c0571132431d
Summary: Being able to push multiple bookmarks in a single hg push is required for using hg push as tailing of fbsource which contains few remote bookmarks
Reviewed By: StanislavGlebik
Differential Revision: D7743737
fbshipit-source-id: ba24445762baafbaa5b3295dc8995fe871f97872
Summary: For on-disk-rocksdb use cases we should persist bookmarks like any other table we use
Reviewed By: farnz
Differential Revision: D7728717
fbshipit-source-id: f63a6410f5ed254a719a16a7504d1b31da5a20a8
Summary:
This change isn't doing much on it's own since rocksdb's BlobRepo is using in memory Bookmarks ATM and they dissapear when import is finished.
Later bookmarks will be used for Head discovery, then it will be properly tested
Reviewed By: farnz
Differential Revision: D7728716
fbshipit-source-id: ad50f35b18d93aa1e38951408092e46e67fde0c7
Summary: I am planning to add importing bookmarks, doing it on current main.rs would make it unreadable, so I am splitting this file now
Reviewed By: farnz
Differential Revision: D7728185
fbshipit-source-id: fdfb4f60eecd9c8af7626bd0e892bb1bfbf7f081
Summary: new blobimport has taken over integration tests
Reviewed By: StanislavGlebik
Differential Revision: D7710186
fbshipit-source-id: 7d61f3239d8edbb1f458dfde0abf59914a912aeb
Summary: The eden integration test contains a commit with no content which new_blobimport couldn't import. With this changes the commit API is capable of handling such commits.
Reviewed By: jsgf
Differential Revision: D7709243
fbshipit-source-id: 7d55eb2ec421820d189ab05b0f8cb4411f850a7b
Summary: The problem in compute_changed_files is that for 2 parents given instead of producing a list of files that were changed in the merge commit from perspective of both parents it produced the opposite of that. This should fix it
Reviewed By: jsgf
Differential Revision: D7708215
fbshipit-source-id: d58457054625beb853f61cb060d25b09279ecceb
Summary:
Let's fail only if inconsistent data was inserted - for example, same commit
hash but different parents.
This matches core hg behavior, and also it's completely normal for commit cloud
to send more parent commits than necessary.
Reviewed By: lukaspiatkowski
Differential Revision: D7722649
fbshipit-source-id: 172a0985fb3fda27d55e9dce8916ec3793de5db9
Summary:
This is a nice little improvement, that makes code readable.
`asynchronize()` does all the job of executing the job on the thread pool.
Reviewed By: lukaspiatkowski
Differential Revision: D7722648
fbshipit-source-id: 6d87178c11fa56ea9738738357fb3c3af012c81a
Summary:
There are a few separate steps during blobimport. One of them is inserting the
blobs and another is inserting the changesets. That worked fine if RevlogRepo was
static. However if RevlogRepo has changed between first and second step we
could've inserted blobs for N commits, but insert (N + X) changesets into
changesets table. That means that a few commits would have no blobs at all.
This diff fixes it by fixing the changesets that we want to blobimport
beforehand.
Reviewed By: farnz
Differential Revision: D7615133
fbshipit-source-id: 1a66907e34a65588b101199c8f59abda53f7bc20
Summary: As with filenodes, we also want to write changesets in a real store.
Reviewed By: lukaspiatkowski
Differential Revision: D7615101
fbshipit-source-id: 269deb8fc3c1f58afb82f453a68ea4d8a3f1f63d
Summary:
Use asyncmemo to cache Changesets.
Unfortunately currently we are using separate asyncmemo cache, so we have to
specify the size for the caches separately. Later we'll have a single cache for
everything, and the number of config knobs will go down.
Reviewed By: lukaspiatkowski
Differential Revision: D7685376
fbshipit-source-id: efe8a3a95fcc72fab4f4af93564e706cd1540c2f
Summary:
Let's use it! Pass config option that set's the cache max memory usage (don't
put a limit on the number of entries, it's useless in that case).
Currently we'll set a separate size for each of the caches that we use
(blobstore, changesets, filenodes, etc). Later we'll have just one single option that
sets the cache size for all of them.
Reviewed By: lukaspiatkowski
Differential Revision: D7671814
fbshipit-source-id: f9571078e6faaa80ea4c31c76a9eebcc24d8a68a
Summary:
Update x86 and aarch64 toolchains, rust-crates-io update.
The `use_nested_groups` feature is now stable, so we don't need to list it in
`#![feature(...)]` lines.
Reviewed By: kulshrax
Differential Revision: D7690730
fbshipit-source-id: 5f74bb01df6bd88a66ac00a82e446c18e3a1c43d
Summary:
The diff size is formidable, however it's not that bad really. This is just
about moving the code inside the macro and tests in a separate file.
This is similar to changesets or filenodes crates
Reviewed By: lukaspiatkowski
Differential Revision: D7639681
fbshipit-source-id: 4216652780bf99939245ae39e508b3b46cf96a03
Summary:
Just as with changesets, let's make filenodes async.
I had to make unit-tests use async_unit, otherwise tokio::spawn() failed with
SpawnedError. I'm not sure why it happened, but async_unit has fixed the issue
Reviewed By: lukaspiatkowski
Differential Revision: D7635554
fbshipit-source-id: c6130b95cee2d60d0b6eec0ab6c1b3e4f3a26ccd
Summary:
This is similar to what we've already done in filenodes and changesets. Sqlite
implementation just grabs the lock, future mysql implementation will get a
connection from pool.
Reviewed By: lukaspiatkowski
Differential Revision: D7585387
fbshipit-source-id: 95a046808f4d78d7776a26bdf3f7d939b1ee0451
Summary:
This is a great suggestion from lukaspiatkowski. This method allows us to take synchronous code, schedule it on the default tokio thread pool and convert it to the future.
The great use case is diesel connections.
Reviewed By: lukaspiatkowski
Differential Revision: D7685244
fbshipit-source-id: ba5a99a7ed977a3aa8b5115049cdd71d9b11112c
Summary: After ceasing recomputation of NodeHashes the newblobimport is working in many of our tests as a replacement of blobimport
Reviewed By: sid0
Differential Revision: D7707684
fbshipit-source-id: e7b4391916cd4a37968afd828f456a7b49ecabf9
Summary:
We know that the hashes for non-root-tree-manifests and filenodes
should always be consistent. Verify that.
Reviewed By: farnz
Differential Revision: D7704087
fbshipit-source-id: 7f6207878c5cd372b272aa6970506dd63b5a3c7c
Summary:
As the comment explains, sometimes the hashes don't match the
contents. Accept such pushes.
Reviewed By: farnz
Differential Revision: D7699930
fbshipit-source-id: 376f01b6cf03f6cad84c2c878d192d55f8d81812
Summary:
* `Into<Bytes>`, not `Into<String>`
* Centralize in one place because I'm going to be adding more non-trivial code to this in upcoming diffs
Reviewed By: farnz
Differential Revision: D7694764
fbshipit-source-id: e6373cf1562b9a464894466ecf3be59e9892ac78
Summary:
We're going to keep this around for now as part of double-writing.
All the hashes here are definitely Mercurial hashes, so use them that way.
Reviewed By: lukaspiatkowski
Differential Revision: D7683890
fbshipit-source-id: 270091cd11f3cec7ef4cf565de5ef913fcf7adea
Summary:
file::File works entirely in the Mercurial domain, so these
conversions are good.
Reviewed By: StanislavGlebik
Differential Revision: D7665973
fbshipit-source-id: 8a192c5d1886492ad21593693b080c8e5ddf8f7e
