Summary:
There have been a bunch of problems with the previous approach to scmstore, so I'm going to try to start simple, make it feature complete, and then add async integration and factor out generic functionality as appropriate.
This change contains a `TreeStore` implementation with a single, synchronous, batch read method (supporting local storage, memcache, and legacy fallback, with writing missing to cache).
Add `TreeStoreBuilder`, which duplicates the existing `TreeScmStoreBuilder`, which some changes that make it easier to use for this case. I intend to unify these in the future.
Add an inherent impl for `EdenApiTreeStore` that provides subset of the `BlockingEdenApi` trait, which eliminates the need to unpack this type into a different adapter as the old `scmstore` code does. This might not be the right approach here, in reality we only need a `(client: Arc<dyn EdenApi>, repo: String)` here for trees, and that + `ExtStoredPolicy` for files, so we could take the `EdenApiAdapter` approach here too. The only reason we have to do any of this is because when `pyrevisionstore` is called to construct `scmstore` / `contentstore`, all we have is `Arc<EdenApiTreeStore>`. We could also just make the `EdenApiRemoteStore` fields public, and access them through the `Arc`.
Add `add_mcdata` method to `MemcacheStore`, `impl TryFrom<Entry> for McData`, and `impl From<McData> for Entry` for convenience when working with `MemcacheStore` (so we don't need to manually unpack the type and build `Entry`, or manually build a fake `Delta` from `Entry` to write).
Reviewed By: DurhamG
Differential Revision: D28076900
fbshipit-source-id: 7fdb5e8a42d052879eff449f60d40a83cfa7145d
Summary:
Both `get_local_path` and `get_cache_path` take suffix as as `PathBuf`, even though they only ever use it as a reference. `get_local_path` also takes a reference to a `PathBuf`, even though it always clones it internally, and takes an `Option`, even though it just maps across the contents of the option.
I modified `get_local_path` to accept a `PathBuf` by move, which it uses directly, and to not take an `Option` (instead just calling `map` externally, removing some unnecessary unwraps), and for both functions to accept `impl AsRef<Path>` for suffix.
Reviewed By: DurhamG
Differential Revision: D28100527
fbshipit-source-id: df28b51c8005f3d95acc8e082b40adaab18e31c9
Summary: Add a Read/Write Guard API to IndexedLogHgIdDataStore which allows client code outside the module to perform a series of reads and writes without locking for each individually.
Reviewed By: kulshrax
Differential Revision: D28075788
fbshipit-source-id: 2a65a426f443e1a421198ad8b4c610e4822574f7
Summary:
Add get_entry, put_entry, and flush_log inherent methods to IndexedLogHgIdDataStore. Refactor callers to use them in cases where they don't lock across multiple reads / writes (to avoid performance regressions).
This should allow `ReadStore` and `WriteStore` to be moved out of the module.
Reviewed By: DurhamG
Differential Revision: D27979828
fbshipit-source-id: c9fb8c4ac68f67b285c72396509aa17928aa54ed
Summary: It has been wrong since 2014 (tweakdefault).
Reviewed By: kulshrax
Differential Revision: D28122703
fbshipit-source-id: c83ddbac2c6162e36672649c60c2e7916dc7cbd2
Summary: This is step towards unifying native merge/rebase structs with native checkout - we now construct native checkout plan from the action map, instead of directly making it from the diff
Reviewed By: quark-zju
Differential Revision: D28078156
fbshipit-source-id: 318d7e419ca9fef15a4aebf7494451f69a3bbbe5
Summary:
This diff makes concurrency of native checkout to be configurable
This config can be used to reduce concurrency on platforms that are known to cause issues with watchman due to too many checkout operations
Reviewed By: quark-zju
Differential Revision: D28074993
fbshipit-source-id: 0a09fcf3ae48d08cead36da56c06b546aecd16b4
Summary: This diff refactors out `Checkout` component from checkout plan and allows to configure parallelism in checkout
Reviewed By: quark-zju
Differential Revision: D28074994
fbshipit-source-id: 72933c757d6e27615d1ef2bb4652bc67c9c3253d
Summary:
Vertex is old. It no longer makes sense with the current structure. The main
issue is that the dag crate now has VertexName which may confuse readers at
first glance.
When Vertex was added DagId would have been confusing because we had structs
that were named Dag that did not use DagId directly. Those structures are now
renamed and DagId is consistently used for dag crate structures.
The IdMap database would still use the vertex name until someone runs a
migration to rename the column.
I am not 100% that this is needed, but it's a change that's been on my mind.
Reviewed By: quark-zju
Differential Revision: D28110184
fbshipit-source-id: b996a7545a90acc25e2bb5326f2731b95c8740b4
Summary:
Previously there were two different paths to HgChangeset. This diff unifies them, so that when walker state.rs is checking for a previous visit it will find that it happened.
For existing walks of changesets in the NewestFirst direction this wasn't causing a problem, however the next diff in stack adds support for OldestFirst walks. In the OldestFirst case the mismatch in paths to HgChangeset was leaving a deferred edge to visit when everything should have been visited in previous chunks.
Differential Revision: D28095569
fbshipit-source-id: ccba4a679fc28bde042cfc222e5097c84fa968c0
Summary:
Right now we write straight to a logger with no filter, so no matter the log
level we print this stuff out. Let's fix it.
While we're at it, move this back to debug level.
I'd made this trace in my recent cmdlib refactoring (which resulted in us
properly initializing logging in all binaries), since I assumed we just had level
filtering working but with debug-logging enabled and I didn't want to have to
update every single test, but it turns out that the reason we didn't print it
out at trace is just because thats not enabled at all in our slog build:
D28097080.
Reviewed By: StanislavGlebik
Differential Revision: D28116053
fbshipit-source-id: f59d9a70ea3c3d834adea16f2686bfc244672b14
Summary: The precise compressed size of big blobs in zstd varies between runs. Glob out the exact size
Reviewed By: StanislavGlebik
Differential Revision: D28116066
fbshipit-source-id: 990add820de6c8cb0029805bc1de304fdf83acba
Summary:
It wasn't in warm bookmark cache, but it was an oversight - there's no reason
for it to not be here. Let's add it, since in case of crashlooping derived data
tailer (see attached task T89911396) there might be nothing to derive fastlog
data structure, and we end up with a long queue to derive.
Reviewed By: krallin
Differential Revision: D28114533
fbshipit-source-id: feb29c07d90be6250c5385ae9f2fb13eb52eedba
Summary:
From what I can see, this was added when EdenFS had a Mononoke store, which is
now long gone, thus we should be able to remove the Curl dependency altogether.
Reviewed By: fanzeyi
Differential Revision: D28037816
fbshipit-source-id: 834f7db64bab5dda1748ad2f033c27a2854b0ba4
Summary: Looks like these aren't needed since these files are owned by a TARGETS file.
Reviewed By: genevievehelsel
Differential Revision: D28101197
fbshipit-source-id: d790530227641bf25e48bd96c8a95dd31f08a954
Summary:
Now that autodeps knows where to find cpptoml.h, we no longer need these
manual annotation.
Reviewed By: kmancini
Differential Revision: D28100956
fbshipit-source-id: 463b73834c500c1d16a4a769af3655938124d49d
Summary:
For no particular reason I was looking at this and saw a bunch of
unneeded `vec![]` temporaries which could be replaced with arrays or slices.
Reviewed By: krallin
Differential Revision: D28073693
fbshipit-source-id: 7fca3b4c7b40cc380b4b128e9809912b7b9ba1f7
Summary:
The original bug that resulted in empty revisions being pulled is long-fixed:
T28553115. I'm planning to make data1 nullable so I can reclaim space by removing older
revs.
Reviewed By: DurhamG
Differential Revision: D28096278
fbshipit-source-id: a57da458df115dcbdf544e2151aa327651190c1a
Summary:
This enlists hgsql tests to the lists of tests using revision numbers and
marks some racy lines as optional
Reviewed By: quark-zju
Differential Revision: D28096282
fbshipit-source-id: eb8406cb74f3338d13d4109fce35f969ff9e3b79
Summary:
This is a hg-sever backport of fix from D27659634 (8e8aaa61d6)
Those are not used. Recently we saw build issues like:
lib/third-party/sha1dc/sha1.c:8:10: fatal error: string.h: No such file or directory
#include <string.h>
^~~~~~~~~~
Possibly by some compiler flags disabling stdlib. Since we don't need
the C code let's just remove them.
Reviewed By: StanislavGlebik
Differential Revision: D28096283
fbshipit-source-id: 6c5390d26264e1e39f99b29dec8608d92e5ae572
Summary: - Like it says in the title.
Reviewed By: HarveyHunt
Differential Revision: D28092796
fbshipit-source-id: 01816f815148aca6c86078fb7dec616ecf53095c