Summary: They will be reused in import_pull_data
Reviewed By: quark-zju
Differential Revision: D29147950
fbshipit-source-id: 192bf33c30067f43c4fcaaf3054741b39efb4e25
Summary: This is an interface for importing pull data into dag
Reviewed By: quark-zju
Differential Revision: D29142979
fbshipit-source-id: b40b94403a044c0b74d1574528aa374ec309a0cf
Summary: This will be used to import pull data into segmented changelog
Reviewed By: quark-zju
Differential Revision: D29142981
fbshipit-source-id: 2d19a035ee0b6cefef8fc0547a5dfb79f284a1de
Summary:
`cl.userust()` is always True after D29020191 (3765f8bd76) so `cl.userust() is False` code
is now dead.
Reviewed By: andll
Differential Revision: D29142464
fbshipit-source-id: f9a7e5c56641218758f12bad3de43d1cd1a71716
Summary: Pack files are no longer supported, yet we still have many tests which exercise them. In preparation for landing `scmstore` as a drop-in replacement for ContentStore, I'm removing our tests which only exist to test datapack-specific functionality.
Reviewed By: DurhamG
Differential Revision: D29099012
fbshipit-source-id: 635a913ee0d93ed8d536e71f8fa6a600b823b343
Summary:
The only real change here is: https://github.com/BurntSushi/ripgrep/pull/1756
This is a patch release but fixes a very glaring bug that others have
depended on. This diff fixes the uses to match the old behavior.
Although it's billed as a "fix", it's actually a huge perf improvement
for Linttool, which uses predominantly recursive suffix globs. The fact
that we don't have to compile ~5,000 regexps at Linttool startup anymore
makes such a huge difference that I am going to do write up soon.
Reviewed By: ndmitchell
Differential Revision: D29085977
fbshipit-source-id: 304470e5fa8cb986738aa0d9dd941641684a9194
Summary:
Python 2 is no longer built anywhere. Let's make the various py3
options the default, like renaming 'make local3' to 'make local' and let's get
rid of the dead setup.py and rpm spec.
Reviewed By: quark-zju
Differential Revision: D29077093
fbshipit-source-id: 0c50c2296fe10ff1db9ac8f9b0df2a4836c0ea5b
Summary:
For a while we had two methods in async runtime: block_on_future and block_on_exclusive, due to historic reasons
Recently those methods were calling same code, and now it is time to replace both of them and rename to block_on
Reviewed By: quark-zju
Differential Revision: D29121107
fbshipit-source-id: 5faa76ae181e491b55d799c23c9de1b4e80298f3
Summary:
If this opens a transaction and fails, then we start a never-ending cycle of
starting hg cloud backup and failing.
This command normally doesn't open transactions but it might via other
extensions, here is one example: P423007962.
This ends up killing people's machines:
https://fb.workplace.com/groups/scm/permalink/3957513017631622/
This fixes that by not kicking off background backups in this case.
Reviewed By: liubov-dmitrieva
Differential Revision: D29136077
fbshipit-source-id: 1c4e6de6147571dd6d728f761324506b1804f303
Summary:
The added command gives access to read (execute) dynamicconfig without
using an on-disk repo.
It can be used by the clone script to stage rollout lazy changelog, or just to
verify dynamic config changes without using a repo.
Reviewed By: DurhamG
Differential Revision: D29123072
fbshipit-source-id: e8856d816a636fa860bfcc9694306a4a37552523
Summary:
implement uploading file content via Eden API
* in this diff I aim to upload file content for the given set of filenodes
* also, the code would check with Mononoke using the lookup request what is already there and skip those
* also, this diff introduces calculation of blake2 hash (called ContentId) for file contents (we would probably need to store/cache those and the mapping from hg filenode id to the canonical Mononoke content_id)
* for every uploaded content EdenApi returns a token that we would also need to store later
Reviewed By: markbt
Differential Revision: D29063229
fbshipit-source-id: 739a44bc3ff904cb04a39514ba5efd01c80ba6d0
Summary: This will be used in eager repo integration tests
Reviewed By: quark-zju
Differential Revision: D29113218
fbshipit-source-id: a24232bd6c19010d8ac90d1305f57f1094b06323
Summary:
Similar to D29111710. Let's avoid asking servers for unknown nodes.
In theory this might affect correctness. Practically, this should only affect "landed as" markers,
because all drafts should be non-lazy. If the "landed as" correctness is an issue, we can fix
forward "landed as" later (ex. by writing down the public commit hash explicitly in
debugmarklanded).
Reviewed By: andll
Differential Revision: D29114049
fbshipit-source-id: db8dc34244feb66919cdff9433b6f18967c9ea9b
Summary: This will be used by upcoming changes.
Reviewed By: andll
Differential Revision: D29114048
fbshipit-source-id: ce963a42ebc5722ca9c61ab0a4e7a7377bbf7abb
Summary:
The mutationstore might contain lots of names (commit hashes) that are unknown
to the repo. When using a lazy changelog, by default, those names will be
looked up remotely, and most of the time the server just returns "name is outside
the graph".
We don't have negative cache to speed it up, because cache invalidation is
tricky - every lazy pull would in theory invalidate them.
To make things faster, let's just skip names in the mutationstore that are unknown
_locally_ without asking the server.
In theory this might affect correctness. Practically, this should only affect "landed as" markers,
because all drafts should be non-lazy. If the "landed as" correctness is an issue, we can fix
forward "landed as" later (ex. by writing down the public commit hash explicitly in
debugmarklanded).
Reviewed By: andll
Differential Revision: D29111710
fbshipit-source-id: 2c1b16a8140ca4f7195bf1206ffe1db4750185b9
Summary: This allows BTreeMap to be used as a provide of config for testing.
Reviewed By: andll
Differential Revision: D28019023
fbshipit-source-id: 94a1585139d6c6612fa163f2ff1aecc909db3a72
Summary: Bumping the crossbeam version so we can use `recv_deadline`. This also necessitates updating the lsp and lsp-types crates.
Reviewed By: alunyov, dtolnay
Differential Revision: D29056473
fbshipit-source-id: 9434e9e0895d82482f4c70afa01a2f77702b965f
Summary:
This functionality is exercised in `test-lfs-copytracing.t` and a few other tests, though I'm not sure if it's necessary outside of that. I've added a `TODO` to investigate and try to eliminate such cases.
Because we'd rather not support this, I implemented it on top of ContentStore - it looks like we might have some lingering datapack-related tests where I think this comes up, too.
Reviewed By: DurhamG
Differential Revision: D29056647
fbshipit-source-id: e6bd5ec31dde06d8c509665f738cfe19270ddc76
Summary:
Modify the `ExtractInnerRef` implementations for `filescmstore` and `treescmstore` to return the new scmstore objects rather than the deprecated ones.
Modify `pycheckout` to use a new methods, `get_oldscmstore`, to access the deprecated scmstore instead (native checkout will be updated to use the new scmstore later on).
Reviewed By: DurhamG
Differential Revision: D29056498
fbshipit-source-id: 09b4655941bca048b01e6d365fcca1ccfb1cd700
Summary: Temporarily implement `upload` on `FileStore` by forwarding to the existing `upload` implementation using `LfsRemote`. Eventually we'll want to re-implement this entirely inside `FileStore`.
Reviewed By: DurhamG
Differential Revision: D29045264
fbshipit-source-id: 8ebbaead4c1032949e5bdd1dcf9a733d0086a77c
Summary: Add a `get_shared_mutable` method to `TreeStore` which behaves like `ContentStore::get_shared_mutable`, returning a store which will read/write only to the local cache stores (aka shared stores).
Reviewed By: DurhamG
Differential Revision: D29042716
fbshipit-source-id: f2236ff8b47ef213b2ffc61501ca301da02dc492
Summary: Add a `get_shared_mutable` method to `FileStore` which behaves like `ContentStore::get_shared_mutable`, returning a store which will read/write only to the local cache stores (aka shared stores).
Reviewed By: DurhamG
Differential Revision: D29042485
fbshipit-source-id: e28af0bafac5eba87523b0ef522f32355106b467
Summary: Add `FetchLogger` to `FileStore` to track remote fetches which match `remotefilelog.undesiredfileregex`.
Reviewed By: DurhamG
Differential Revision: D29042243
fbshipit-source-id: 08ec69d979d66ef93e8bbe8659171a412ade7e22
Summary: `ReportingRemoteDataStore` wraps a store and records all fetches which match `remotefilelog.undesiredfileregex`. Since scmstore doesn't use a hierarchical store composition, this change extracts out the matching/reporting logic into `FetchLogger` so that it can be re-used by both `ReportingRemoteDataStore` and scmstore.
Reviewed By: DurhamG
Differential Revision: D29041488
fbshipit-source-id: db0c4b545886ffebcf2a2841a506d301b2f2f230
Summary: Straightforward update of `debugscmstore` from old scmstore to new scmstore. I'll want to improve this command a bit more in the future, but this at least enables you to easily test arbitrary fetches.
Reviewed By: DurhamG
Differential Revision: D29047527
fbshipit-source-id: 4e10cb88cba4b572d3e413640ca3d800940d675d
Summary:
Our log deletion is a two step process. First it deletes the meta file,
then it deletes the log. There was a bug where if the meta file deletion
succeeded but the log deletion failed (often because Eden has the log open) then
future attempts to delete the log will fail because it fails to delete the meta
file.
Reviewed By: quark-zju
Differential Revision: D29035913
fbshipit-source-id: 33ab9a4e46b285819c4bf14031426983d2b4342f
Summary:
Since all the errors are eaten, we don't have good insight into the
rotatelog cleanup process. Windows users were seeing large growth in their
hgcache and this logging would've been useful, so let's add it.
Reviewed By: quark-zju
Differential Revision: D29035914
fbshipit-source-id: 54f0f05aa8bed1911b6b95438cd2d3ccc45e8ec9
Summary:
The crecord width calculation used a function that requires a string,
so we tried to force the rows from bytes to columns, which crashed if the row
wasn't utf8. Let's just use the error="replace" utf8 error handler since we
don't care if the row computation is slightly off.
Reviewed By: quark-zju
Differential Revision: D29043385
fbshipit-source-id: a00d867bca9c62007ae4f68b6198d159deeb6016
Summary: This method will be used to get all Ids that needs to be included into CloneData::id_map during fast forward pull.
Reviewed By: quark-zju
Differential Revision: D29045538
fbshipit-source-id: f6eb04f537aa5365f7588391ea4c60b3dc010a53
Summary: revert the zstd crates back to previous version
Reviewed By: johansglock
Differential Revision: D29038514
fbshipit-source-id: 3cbc31203052034bca428441d5514557311b86ae
Summary:
implement `known` method based on the existing `lookup` API that has been already implemented
this might be useful for eden api pull project / wireproto deprecation, as the pull algorithm queries the server for whether commits are known by the server
Differential Revision: D29030429
fbshipit-source-id: 4d78c8b1e7983d89de917951d9765b1bd1d90fb0
Summary: This will be used in fastpath pull
Reviewed By: quark-zju
Differential Revision: D29010304
fbshipit-source-id: 9a7464fbc42544f8c5cc52f6fa25db5f7b5d6bd9
Summary: idset_to_flat_segments allows to return intersection between flat segments and span set
Reviewed By: quark-zju
Differential Revision: D28980521
fbshipit-source-id: da90c66a021a1e8bb95edbbc2e06cd2a850d7dcd
Summary:
This diff introduces Subspan trait for objects that contains spans and allows to take a 'subspan' of an object
Implementing Subspan trait allows to intersect arbitrary objects and spans
Reviewed By: quark-zju
Differential Revision: D28980523
fbshipit-source-id: 29a2e58ec3c79a838db751d8a37227b4cdaaa372
Summary:
It is only useful for Python revlog-based changelog, which was removed by the
previous diff.
Reviewed By: DurhamG
Differential Revision: D29020190
fbshipit-source-id: 3332f1532a510dc72f21d68801e093b5eba10145
Summary:
Modify the status command to prefer comparing content sha256 when comparing files over comparing content directly, using cached aux data when possible and writing newly computed hashes to cache.
A new config option, `scmstore.status`, gates this new behavior.
When `scmstore.status` is enabled, the `status` output should be more accurate, not assuming files with non-matching `HgId`s in an arbitrary pair of revisions are different. As written, this will cause fetching of data that wouldn't otherwise be fetched, but remote fetching of aux data is planned.
This change also doesn't introduce batch fetching, this is planned.
Reviewed By: DurhamG
Differential Revision: D28826219
fbshipit-source-id: dcd253f169723983738148559d61ba655dd453a4
Summary: Introduce a config for enabling separate indexedlogs for storing "aux data" in scmstore. The storage format is not yet finalized, and isn't used in production yet, so we gate the constructing it on a cofig.
Reviewed By: DurhamG
Differential Revision: D28826209
fbshipit-source-id: 1ed8c20329f21b3ced598eea16d9fac0f40112e6
Summary:
Make `doctor` aware of non-revlog changelog and do not hide commits
incorrectly.
Reviewed By: DurhamG
Differential Revision: D29020189
fbshipit-source-id: af7fab75a877997e3f11dd21e506c965da078f83
Summary: The doctor command assumes revlog changelog, which does not cover all setups.
Reviewed By: DurhamG
Differential Revision: D29020188
fbshipit-source-id: 68bb4e20efb49482df84a71ab7d86a20a6fa93ce
Summary: For all clients and tests we're using revlog changelog based on Rust.
Reviewed By: DurhamG
Differential Revision: D29020191
fbshipit-source-id: 44ab603ea07b8891831692285e94ea0e4df849f5
Summary:
Verify is tightly coupled with revlog details, and assumes a non-lazy repo.
Revlog is going away and our repo is lazy typically, so let's just make verify
a no-op.
Reviewed By: DurhamG
Differential Revision: D28974546
fbshipit-source-id: 487dc65ceb9a1e445b58d03a53353a8382aefc08
