Summary: These tests were broken in D27710099 (876f812e4b), but they show as passing unless run in a particular environment, so it went unnoticed. This change reverts the tests to use the pre- D27710099 (876f812e4b) behavior, which should unbreak them until they can be updated correctly.
Reviewed By: quark-zju
Differential Revision: D27756348
fbshipit-source-id: cfa6c12871b6ac0d22b8c70400e72b3ec5dd83a3
Summary:
The `add_heads_and_flush` method might add new nodes in the master group,
and it should update `overlay_map_next_id` accordingly. Without it, it
might error out like:
RustError: ProgrammingError: Server returned x~n (x = 9ebc9ebc49f1819767b40f9ceb22c37547a10c37 8459584, n = 1411).
But x exceeds the head in the local master group {}. This is not expected and indicates some logic error on the server side.
Full error: P387088806
Reviewed By: sfilipco
Differential Revision: D27637278
fbshipit-source-id: b45370db0561dec52cd513a12e2fd0110f18e0e5
Summary:
The filternodes is an API that can batch hasnode checks. It is more efficient
if the commit hashes are lazy.
Reviewed By: sfilipco
Differential Revision: D27636338
fbshipit-source-id: 4eb2dcd20b939faa38611b82e32ed97cf0c8f175
Summary:
The filternodes is an API that can batch hasnode checks. It is more efficient
if the commit hashes are lazy.
Reviewed By: sfilipco
Differential Revision: D27636341
fbshipit-source-id: 69cd708a46c719624d654c53de3d92968392d572
Summary:
If a vertex was resolved via remote protocol, respect it and
avoid requesting the same thing twice.
Reviewed By: sfilipco
Differential Revision: D27636340
fbshipit-source-id: 43cf86010745a85cf622c67be4261ea47a33c802
Summary:
Many places use the `[n for n in nodes if hasnode(n)]` pattern, which
can be slow with a lazy graph due to N+1 problem. Add an API so the
Python world can use a more efficient way for the same query.
Reviewed By: sfilipco
Differential Revision: D27636339
fbshipit-source-id: 99ccb85b2266aed22f1cf87a5e2528106edb3783
Summary:
That could cause a slow loop testing node.__contains__ remotely.
This changes the behavior subtly - nodes added via addgroup will change `tip`
position regardless of whether the nodes exist. This might be more desirable,
since add or addgroup explicitly adding nodes should probably update the
tip position.
Offending test `test-globalrevs-requires.t` was removed since we have
forked the server-side codebase and do not need to maintain hg server
features here.
Reviewed By: sfilipco
Differential Revision: D27630090
fbshipit-source-id: cf7ecc44bf08ed756f0f1aece6655bf674171249
Summary:
The idset is not fully backed by Rust and do not batch resolve vertexes.
The nameset is backed by Rust NameSet and has proper batch prefetching.
Use nameset if possible but fallback to idset if the backend is not in
Rust (rare, only used by hgsql repos now).
Reviewed By: sfilipco
Differential Revision: D27630092
fbshipit-source-id: cf847dd1a78bd5273a8928ecb6616fe11f2c7026
Summary:
This will be useful to avoid suboptimal code paths that is very slow
with lazy vertexes.
Reviewed By: sfilipco
Differential Revision: D27630089
fbshipit-source-id: 35ee4ba79b551453de78fd22aecccf10bc43b08b
Summary:
While it is in theory "correct" going through the remote resolution even if the
protocol is "local". The overhead turns out to be something. And the tracing
message "resolve .. remotely" can be quite noisy. Let's just skip remote
resolutions early in IdConvert implementations.
Reviewed By: sfilipco
Differential Revision: D27630094
fbshipit-source-id: 7d87079876f040cf8f764f7362021dddba0d4723
Summary:
Currently the "contains vertex" check can trigger excessive
fetches for add_heads (and add_heads_and_flush used by flush).
Add a test to demonstrate the problem.
Reviewed By: sfilipco
Differential Revision: D27630091
fbshipit-source-id: ce3639c2a13226ba5681b4e8696edd7acbcb57f9
Summary:
Otherwise it can cause a lazy dag to think vertexes as "missing", insert
vertexes unnecessarily, and potentially break key graph properties (a
vertex should only have one Id).
Reviewed By: sfilipco
Differential Revision: D27629315
fbshipit-source-id: 1688d13cb94015bbc675613ecf9225556ff48372
Summary:
Also move related functions.
Similar to D27547584 (af3c3b3fd0), this allows `add_heads_and_flush` use `IdConvert`
on the `NameDag`, instead of the `IdMap` to trigger remote fetches properly.
This diff is easier to view with whitespace changes ignored.
Reviewed By: sfilipco
Differential Revision: D27629314
fbshipit-source-id: 8f79223c5d324aabfc5ab9813a9f65400fc533ec
Summary:
See the previous diff for context. Drop Locked and related APIs (prepare_filesystem_sync).
This makes it easier to use operate on a mut NameDag on flush because it does not need
to use separate types (Locked) for writing which has issues of not having the remote protocol
involved.
Reviewed By: sfilipco
Differential Revision: D27629306
fbshipit-source-id: 301445b242321ad5f424741ea91ebf6c075bff1c
Summary:
See the previous diff for context. Drop SyncableIdMap so we are one step
closer to using mut NameDag directly on add_heads, which knows when and how to
do remote fetching properly.
Reviewed By: sfilipco
Differential Revision: D27629310
fbshipit-source-id: 883606e40bb83907dfa6142ddd2c3030de61698f
Summary:
By using SyncableIdDag and SyncableIdMap, it's harder to use extra features
around them (ex. remote fetching). Drop SyncableIdDag so we are one step
closer to using mut NameDag directly on add_heads, which knows when and how to
do remote fetching properly.
Reviewed By: sfilipco
Differential Revision: D27629307
fbshipit-source-id: 8e9a5a4348a42b9751752b82feb3f3d2d7c4ba45
Summary:
The `Parents` trait is used for input of adding (non-lazy) vertexes to the
graph. The API will be used to provide extra hints to optimize network
fetches.
With the current logic, `assign_head` will ask the server to resolve the heads
first, to check if it is already assigned, then to resolve the parents, etc. to
the roots (in the "to assign" set). Ideally the `assign_head` logic can ask
the server to resolve the roots first, and if that's unassigned, then just mark
all descendants of the roots as unassigned, do not send more requests.
Note: the current pull logic has all the hashes ready (hashes are known).
But whether the hashes have Id assigned are unknown. It is more tricky
taking the "lock" and "reload" into consideration - hashes without Ids might
"become" having ids assigned after we obtain the lock to write data to disk.
Practically, `pull` using the current logic would take a very long time because
it tries to resolve things remotely for every "to assign" commits.
Reviewed By: sfilipco
Differential Revision: D27629317
fbshipit-source-id: e02f54f43ef65228ce6e3a8a8723dd9ae0a07008
Summary: This just simplifies the test code a bit.
Reviewed By: sfilipco
Differential Revision: D27629308
fbshipit-source-id: 04eac5cd045c71123e7fc410af74609dbadb8fb7
Summary: This avoids triggering remote lookups if an unknown name was looked up multiple times.
Reviewed By: sfilipco
Differential Revision: D27629316
fbshipit-source-id: 64c1ce5e872a5ce4f14c650a946ae8396f4cc74c
Summary:
When translating RequestNameToLocation to ResponseIdNamePair. If "heads" are
known, but some "names" aren't. Do not treat it as an error. This will be
used by the client-side to properly handle the "contains" check.
Reviewed By: sfilipco
Differential Revision: D27629309
fbshipit-source-id: 206ec5df956b33f4e816ab8d6a67ce776fd7bd74
Summary: This will make it easier to test client / server dags in upcoming changes.
Reviewed By: sfilipco
Differential Revision: D27629318
fbshipit-source-id: e3137654613aa3208a8f2e4b9f4ddfe73871f2c5
Summary: This will be used in upcoming changes. It just delegates to the Arc inner.
Reviewed By: sfilipco
Differential Revision: D27629313
fbshipit-source-id: ba6cd7cac2b9f5c1a2898c439c53768995a2dc42
Summary: This will be used by upcoming changes.
Reviewed By: sfilipco
Differential Revision: D27629312
fbshipit-source-id: 6c56e73caf4e1a398ac3a8b4294bd9f380af3764
Summary: This will be used by upcoming changes.
Reviewed By: sfilipco
Differential Revision: D27629319
fbshipit-source-id: d19e490268561f3154642e5bb1e415da4c5d03ee
Summary:
See the previous diff for context. A concrete HashMap can provide
"hint_pending_subdag". But a parent function cannot.
Reviewed By: sfilipco
Differential Revision: D27629311
fbshipit-source-id: 65168a8d00d9a672396312200016d6749f416d02
Summary: The lazy backend can now (partially) support the non-full IdMap segment clone.
Reviewed By: sfilipco
Differential Revision: D27581488
fbshipit-source-id: 51eded6acdbe82d22f5bb73eb4a715e2c22f4d75
Summary:
Make mutationstore more friendly to async.
This resolves an issue with smartlog with the lazy commit hash backend.
Reviewed By: sfilipco
Differential Revision: D27583844
fbshipit-source-id: 5b0b0b9b8ab82399f80eb2b410a0c4b84bd6a444
Summary:
Otherwise it might panic (ex. calling into tokio without entering a tokio
runtime). This can happen in, for example, `Debug::fmt(&IdStaticSet, ...)`.
Reviewed By: sfilipco
Differential Revision: D27581487
fbshipit-source-id: feec53e088706adcc6710afcf24fa70598f886cf
Summary: `SyncNameSetQuery` will stop working with lazy commit hashes. Change them to async.
Reviewed By: sfilipco
Differential Revision: D27581486
fbshipit-source-id: bfac1d0676f1fe102c74cc4fc2b83d4c9aed970e
Summary:
This will be used by "add_heads" logic to detect what vertexes to insert
and might reduce remote fetches.
Reviewed By: sfilipco
Differential Revision: D27572359
fbshipit-source-id: d0bf027a69d180663a1587dfde613cb76b05072a
Summary: The API returns entries buffered in memory not persisted.
Reviewed By: sfilipco
Differential Revision: D27572360
fbshipit-source-id: 555988f7c891f2d928bfa1e486a0fc1d089b4ad5
Summary: This will be used to select "dirty" (not written to disk) set in the IdDag.
Reviewed By: sfilipco
Differential Revision: D27572361
fbshipit-source-id: 0b4d2e092ece835e3d4b6aa831d32ffffc7087ca
Summary:
Before this change, overlap IdMap was not considered for prefix lookup. That
results in "shortest" template not working and smartlog prints full hashes
for remote/stable etc.
Reviewed By: sfilipco
Differential Revision: D27547582
fbshipit-source-id: 7a56590775eed9d509f2212f8e5009c04aaf4e9d
Summary: It will be reused in NameDag.
Reviewed By: sfilipco
Differential Revision: D27547583
fbshipit-source-id: da85fc7504d20877210e8ed1a97cbec18d06eede
Summary:
Now NameSet iteration can be blocking, SyncNameSetQuery is no longer sound.
Remove SyncNameSetQuery in key logic (namedag and ops) and replace them with
async logic.
Reviewed By: sfilipco
Differential Revision: D27547581
fbshipit-source-id: af69b1a8219e97d10278939407ee79f9b333a77f
Summary: Dag algorithms like `parent_names` need to fetch vertexes via remote automatically.
Reviewed By: sfilipco
Differential Revision: D27547584
fbshipit-source-id: 8106931d6f642c9a4bf0f3c546ba881c2ca73fea
Summary: Similar to "lazytext" but IdMap is also lazy.
Reviewed By: sfilipco
Differential Revision: D27547579
fbshipit-source-id: 70452f1a8e7f00d6a216a2aaec2d55442130d3ce
Summary: This can be used to create lazy commit hash backends.
Reviewed By: sfilipco
Differential Revision: D27547580
fbshipit-source-id: 3329854f1173b8f15fd6b51f4e595d2226c8bbb1
Summary:
This allows unix sockets to be created in the mount. This will allow Buck to
run properly as it tries to create sockets in the repository.
Reviewed By: kmancini
Differential Revision: D27690406
fbshipit-source-id: 5725d68bdda12f3a5882ce48b6bdd02b14cdece4
Summary: This merely adds the types for the procedure
Reviewed By: kmancini
Differential Revision: D27690405
fbshipit-source-id: b94fb03658cabaece4166c29135c5fdf9a613d3c
Summary:
This is roughly the same logic as the UNLINK one with the only difference being
in the handling of "." and "..".
Reviewed By: kmancini
Differential Revision: D27684716
fbshipit-source-id: 86a95c38e6c783bc3a45c0a8b000d0210b6dd0b8
Summary:
This merely adds the types needed for the RMDIR procedure. Implementation will
follow.
Reviewed By: genevievehelsel
Differential Revision: D27684736
fbshipit-source-id: 84f5a4f3dc805e7893853b0de1dc19cb01c1319f
Summary:
To get the size in bytes, we need to multiply the quantity with the block size,
not with itself.
Reviewed By: genevievehelsel
Differential Revision: D27690857
fbshipit-source-id: 7d7ca767881b1118fc24befed230a63f342bc911
Summary:
In revset limit(a, x, y), both x and y are numbers, not commit identities.
The issue is that the revset AST uses different ways to represent functions
with one argument or multiple arguments. For example:
(func (symbol parents) (symbol master))
(func (symbol limit) (list (x) (symbol 1) (symbol 2)))
Fix it by special handling the `list` AST.
Reviewed By: DurhamG
Differential Revision: D27632395
fbshipit-source-id: 081506bdd4b10e197a2685f4ab4d6448fbd79957
