Summary:
This is fairly mechanical diff that finalizes split of Hash into ObjectId and Hash20.
More specifically this diff does two things:
* Replaces `Hash` with `Hash20`
* Removes alias `using Hash = Hash20`
Reviewed By: chadaustin
Differential Revision: D31324202
fbshipit-source-id: 780b6d2a422ddf6d0f3cfc91e3e70ad10ebaa8b4
Summary: - Repo has a main branch now and not master
Reviewed By: xiangxu1121
Differential Revision: D31313971
fbshipit-source-id: 70d151e8a8632e296f25baf07ff9ac8348da7497
Summary:
A spawner type is required for new thrift clients, specify the noop one for now.
This also requires regenerating the generated thrift libraries.
Reviewed By: yancouto
Differential Revision: D31338518
fbshipit-source-id: cbecf3ec6f9678918ca459c19f1cc160214fadfd
Summary:
We now use post-pull hook to mark commits as landed:
hooks.post-pull.marklanded=hg debugmarklanded
So there is no need to mark them inside the pull command. In fact, it seems
something is wrong (phases aren't invalidated properly?) so that the
pullcreatemarkers logic might actually hide commits during pull incorrectly:
commit 1e964a4302c03e5ae48e5b85b0fc0bf27f847b09
Author: metalog <metalog@example.com>
Date: Tue Sep 28 17:29:49 2021 +0000
pull
Parent: a4511a83cf862cc7216c15d83a3f4ff9d3b3241b
Transaction: pullcreatemarkers
RootId: 18d81a1531ecea65affe83c25804c790cac57c59
diff --git a/visibleheads b/visibleheads
index eb58137..da5f45a 100644
--- a/visibleheads
+++ b/visibleheads
@@ -1,16 +1,6 @@
v1
-e82de77adcc261cb306dafeb6cbe15f26f7de768
-91cf8c4b47e433acbe3f774e608eee42a3ad089d
8c071e5aa26d920f1b88c4b1cd10f6e946d4312a
536ff436cde4ec53d74c02ae2c5ed6f60609e01a
-e49b834a6ff9b61a95a743d22703dc6634f2918c
-c53b65542d4583ae82835144ac7f72dce7a6f335
1010312ed7ac683c5e97ad765cdbcb4927ddf62c
-a563a5f35d2df4c54ba1fe2401aa1cd929a218bf
-55c0c7483c6cd85114a9be587d11c87aaceaeff4
-74079567b677c5799b6c67855683ec346ebc3cce
33b5fd6055da2284cf5224f70e1bb9791ed87641
-ec221dff2393793f7b1e11ea9f9e9ea87b79da1f
-cc466614d43a6af24520b7a81653435f4a614fbb
2d7466be885e757d2d41630a3148fd31f5199ffa
-226c8136603dcbcc408133a2122e93fc045527fa
As we're here, document some other config options existed in the code.
Reviewed By: DurhamG
Differential Revision: D31295709
fbshipit-source-id: e26c728215a209ab5dfaee7a84daece8197a1cc4
Summary:
With the lookup processor now returning an ImmediateFuture, we can focus on its
caller, starting with getChildRecursive.
Reviewed By: chadaustin
Differential Revision: D31283396
fbshipit-source-id: 97abc57b9efe3540c5770aa952995c257e6eda4b
Summary:
While profiling glob queries sent by Buck, I noticed that EdenFS spends almost
as much time resolving the inode to perform the glob on as EdenFS takes to
actually perform the glob. Futures related overhead shows up as predominent,
thus let's convert these to ImmediateFuture to speed this up.
Reviewed By: chadaustin
Differential Revision: D31283395
fbshipit-source-id: 7355ddf7498f722ed8ec2989f010a28fb15c293f
Summary:
While std::variant is convenient, they are both slow to compile, and the
compiler cannot optimize it as well as a manually written tagged union. Since
ImmediateFuture is performance critical for EdenFS, let's use a tagged union
and speed them up by an additional 40%.
Reviewed By: chadaustin
Differential Revision: D31272296
fbshipit-source-id: e34be4489a596d3577b3bd900a1f20d6c7d8b693
Summary:
The max duration would cause UBSAN failures due to folly's SemiFuture code
multiplying the value which understandably cannot be represented. Splitting the
function is easy and avoids the problem entirely.
Reviewed By: genevievehelsel
Differential Revision: D31272297
fbshipit-source-id: c15ca70ad771c11b4f68bb9974422c0986d4928b
Summary:
When Buck is using the EdenFS globber, the searchRoot argument is set, thus
let's add a new argument to the benchmark to simulate a Buck workflow.
Reviewed By: chadaustin, genevievehelsel
Differential Revision: D31283399
fbshipit-source-id: 5e32b2aceb6090e26e88cf7f0d163448d56107d4
Summary:
The goal of this stack is to remove Proxy Hash type, but to achieve that we need first to address some tech debt in Eden codebase.
For the long time EdenFs had single Hash type that was used for many different use cases.
One of major uses for Hash type is identifies internal EdenFs objects such as blobs, trees, and others.
We seem to reach agreement that we need a different type for those identifiers, so we introduce separate ObjectId type in this diff to denote new identifier type and replace _some_ usage of Hash with ObjectId.
We still retain original Hash type for other use cases.
Roughly speaking, this is how this diff separates between Hash and ObjectId:
**ObjectId**:
* Everything that is stored in local store(blobs, trees, commits)
**Hash20**:
* Explicit hashes(Sha1 of the blob)
* Hg identifiers: manifest id and blob hg ig
For now, in this diff ObjectId has exactly same content as Hash, but this will change in the future diffs. Doing this way allows to keep diff size manageable, while migrating to new ObjectId right away would produce insanely large diff that would be both hard to make and review.
There are few more things that needs to be done before we can get to the meat of removing proxy hashes:
1) Replace include Hash.h with ObjectId.h where needed
2) Remove Hash type, explicitly rename rest of Hash usages to Hash20
3) Modify content of ObjectId to support new use cases
4) Modify serialized metadata and possibly other places that assume ObjectId size is fixed and equal to Hash20 size
Reviewed By: chadaustin
Differential Revision: D31316477
fbshipit-source-id: 0d5e4460a461bcaac6b9fd884517e129aeaf4baf
Summary:
https://pxl.cl/1Qh3j
This is the most called edenapi endpoint by far. If we sample logging of it, we can increase the retention of the scuba table.
if we wish, it's possible to not change retention for some "non-trivial" requests, but I haven't done that.
Reviewed By: liubov-dmitrieva
Differential Revision: D31277391
fbshipit-source-id: ee19e9daa4cd39c5d3eac1063e82aa40fc108bc7
Summary:
This is used to uniquely identify requests in gotham. It's logged to output, on errors, and on Scuba.
Problem: On scuba, this packs very badly, as it is a large string (36 chars), unique for all requests.
Solution: Let's get a prefix of it, it should reduce size used on scuba. Got a prefix of size 5.
This affects both LFS and EdenApi.
Pros:
- Reduces size
- Very easy fix
Cons:
- More chance of conflict. The space of this id is 16^5 = 10^6. There will surely be conflicts, but maybe that's not a huge deal?
Alternative: Using 8 digits, that's about 4bi ids, which will reduce conflicts significantly in exchange for more space.
Why not use an int id (example: u64), or using other characters in id (not only hex): This would reduce the size of data significantly, but has drawbacks:
- For int, would require a big refactoring, as everything assumes the id to be string. Specially since this goes through client-server, might be complicated.
- Not just getting a prefix means more processing on each request, and means we need to recalculate it everytime.
- Size reduction might not be that big, as scuba already packs stuff pretty well.
Reviewed By: krallin
Differential Revision: D31305547
fbshipit-source-id: 23f6b6cb7de5b7a090864db414d4d71cd68c4946
Summary: D31115820 (ae87b82eaf) updated quickcheck, but there's some stuff we need to fix forward. This diff fixes the remaining failures I could find.
Reviewed By: farnz
Differential Revision: D31305392
fbshipit-source-id: a6684d47833bc0fd933751c13cdd71392cb1833b
Summary:
use `hg cloud upload` command if usehttpupload enabled
For few users who opted out of cloud sync, we should use `hg cloud upload` command instead of `hg cloud backup` if usehttpupload enabled
Reviewed By: markbt
Differential Revision: D31205919
fbshipit-source-id: 7619e7b299e19a7626782e7b3c1a69e7cd7dbc1b
Summary:
Somehow before this fix I've seen us runing out of semaphores and deadlocking
because not freeing semaphores immediately after finishing running the
function requiring git repo.
Reviewed By: mojsarn
Differential Revision: D31310626
fbshipit-source-id: ba12b2d4918ecc30ca0aa6ff011176f7634badf9
Summary: Need this for cargo check/rust-analyzer to work.
Reviewed By: guswynn
Differential Revision: D31319911
fbshipit-source-id: ebd3fa72d8fc3667391a2067f95cab9e5f53301f
Summary: I'm parsing some deeply-nested JSON, and it's running into the limit. This feature enables a potential footgun, but even with the feature enabled you have to add code to reach for said footgun.
Reviewed By: jsgf
Differential Revision: D31284743
fbshipit-source-id: 00ea5d7d7db8bdeb878d48fe390831f39e007409
Summary:
When calling `changelog.idmap`, it takes a snapshot of the IdMap.
When flushing the changelog, the IdMap might change. Some Ids might get
re-assigned.
For code (phases?) that keeps the `nameset` for a while the idmap might become
out-dated after a changelog flush. That might be a cause of removing visible
heads incorrectly.
Let's change `_torev` and `_tonode` to properties so they always take the
"latest" idmap snapshot.
Reviewed By: DurhamG
Differential Revision: D31296057
fbshipit-source-id: 1b3fec5d21649eab772ab3a150a3182a18b94edf
Summary:
It was added on the client side in D30686450 (7eb11cb392) to handle octopus merges correctly,
let's add it on mononoke as well, otherwise new_streaming_clone fails to parse
a revlog.
Reviewed By: mitrandir77
Differential Revision: D31305651
fbshipit-source-id: 976d7fdb8775f859e4732fd8a68f9b28f04ce4f9
Summary: This implements unix socket support for mercurial's HTTPConnection so commitcloud can use it.
Reviewed By: ahornby
Differential Revision: D31229256
fbshipit-source-id: a610c3c34be608ac2d9b41f3a7b6b62b44227b94
