Summary: This wants to use Scuba so it needs this.
Reviewed By: StanislavGlebik
Differential Revision: D28282511
fbshipit-source-id: 6d3a2b6316084f7e16f5a2f92cfae1d101a9c2d3
Summary:
This makes it easier to see what builder functions were registered:
% EDENSCM_LOG=edenapi=debug lhg log -r .
May 06 16:40:29.355 DEBUG edenapi::builder: registered eagerepo::api::edenapi_from_config to edenapi Builder
Reviewed By: DurhamG
Differential Revision: D28271366
fbshipit-source-id: f6c7c3aa9f29c3e47c2449e3d5fc16474aa338b0
Summary:
Adding support for the stables template keyword in stablerev extension.
This keyword calls out to a script specified in the config stablerev.stables_cmd to get a list of stable aliases for a given revision.
Reviewed By: quark-zju
Differential Revision: D28204529
fbshipit-source-id: 3c5b21846ce6f686afddd00d3326a54b85be87dd
Summary:
The server1 was not used after D27629318 (ba7e1c6952) while the test intentionally wants to
exercise graph isomorphism. So let's revive server1 in the test.
Reviewed By: andll
Differential Revision: D28269926
fbshipit-source-id: 0a04031415f559f8a6eb81f1e2f2530329a2a3bc
Summary:
In blobstore factory we can end up with duplicate layers of wrapper blobstores like ReadOnlyBlobstore.
For the multiplex, its inner stores get throttling, readonly etc wrappers, and it itself only writes to its queue if an inner store succeeds, which it can't when inner store has ReadOnlyBlobstore wrapper.
Differential Revision: D28250832
fbshipit-source-id: 5a3f85584b9cce17ca7ce4b83cdb2117644850db
Summary: Add support for logging the inner parts of a multiplex blobstore stack. This helps understand what wrappers have been applied.
Differential Revision: D28230927
fbshipit-source-id: 873ee30ec00fdc2dfc79b47e5831231c51e2ce0d
Summary:
Fixing the bulkops fetch size to MAX_FETCH_STEP means we can use the chunk size option to control how many changesets are walked together without affecting query performance.
This will allow more accurate first referencing commit time lookup for files and manifests, as all commits in a chunk could possibly discover them, with smaller chunks the discovered mtime becomes less approximate, at the possible cost of some walk rate performance if one was to run with very small chunk sizes.
Differential Revision: D28120030
fbshipit-source-id: 0010d0672288c6cc4e19f5e51fd8b543a087a74a
Summary: Knowing the numeric changeset id is useful in next diff when chunking in walker is loading from bulkops in large chunks, but then walking commits in smaller chunks.
Differential Revision: D28127581
fbshipit-source-id: c5b3e6c2a94e33833d701540428e1ff4f8898225
Summary: Found this useful while debugging the pack sampling
Differential Revision: D28118243
fbshipit-source-id: d94b0b87125a9863f56f72029c484909a3696329
Summary:
We were only incrementing this on `readline`, which resulted in very low
numbers. While in there, I also removed `self._totalbytes` as that was unused.
Reviewed By: johansglock
Differential Revision: D28260141
fbshipit-source-id: 6d9008f9342adaf75eecc8ed8c872f64212cd1f7
Summary:
In an NFS mount, the InodeNumber are sent to the client as the unique
identifier for a file. For caching purposes, the client will issue a GETATTR
call on that InodeNumber prior to opening it, to see if the file changed and
thus whether its cache needs to be invalidated.
In EdenFS, the checkout process does unfortunately replace file inodes
entirely, causing new InodeNumber to be created, and thus after an update, an
NFS client would not realize that the content changed, and would thus return
the old content to the application. To solve this, we could approach it in 2
different ways:
- Build a different kind of handle to hand over to the NFS client
- Keep InodeNumber constant during checkout.
After trying the first option, it became clear that this would effectively need
to duplicate a lot of functionality from the InodeMap, but with added memory
consumption. This diff attempts to do the second one.
Reviewed By: chadaustin
Differential Revision: D28132721
fbshipit-source-id: 94d470e33174bb9ffd7db00e1b37924096aac8e9
Summary:
Add a subtree to exercise treemanifest logic. Blobs in EagerRepo are verified
so we need to disable flatcompat.
Reviewed By: DurhamG
Differential Revision: D28006550
fbshipit-source-id: ac7157a9c01ed99f703601613fb3cf06add69003
Summary: This makes it easier to use it in tests.
Reviewed By: DurhamG
Differential Revision: D28006549
fbshipit-source-id: 90e29b220453a3d7a260d0a62d697d64363d9a6c
Summary:
With remotefilelog force enabled, it's now possible to read the file content after
clone. Add a test for it.
Reviewed By: DurhamG
Differential Revision: D28006547
fbshipit-source-id: 5be93e162f352b1264a6c52852c2230726652f9d
Summary:
This makes it easier to get rid of revlog stores.
`debugindexdot` is no longer supported since it reads revlogs.
Two tests use flat manifest bundles. They are no longer supported
due to remotefilelog today has some assumptions that treemanifest
extension is also being used.
Reviewed By: DurhamG
Differential Revision: D27971126
fbshipit-source-id: fdb992a8d72bbcf562b5cb95b3f29051dd1c9464
Summary:
Disabling treemanifest is a tech debt that causes problems, especially when
enabling remotefilelog.
Reviewed By: andll
Differential Revision: D27971120
fbshipit-source-id: 1a50acc23564c2d6bad79a2e99469850b5a7d1f9
Summary:
This makes it easier to filter logs related to remote fetching.
The `DEBUG dag::protocol: resolve ids [0] remotely` means the lazy hash resolution is working.
Reviewed By: kulshrax
Differential Revision: D27971117
fbshipit-source-id: f2492204c70d793997d0c3865e500bbad56b1953
Summary:
Write commit to master group. This provides proper "CloneData" and allows us to
actually test lazy commit hash backend (since only commits in the master group
can have lazy hashes).
Reviewed By: DurhamG
Differential Revision: D27971123
fbshipit-source-id: 4e19486007ddc89de7468be65445559f34d796f5
Summary:
Add clone endpoint so we can clone from an eager test repo.
Note: the master group is empty and "cloneata" does not quite work yet due to
EagerRepo not writing to the master group. It will be fixed later.
Reviewed By: DurhamG
Differential Revision: D27971121
fbshipit-source-id: 0cc35136c6987673c2c4fbbd892c344c3586fcb7
Summary: This update makes it so that we don't log versions to scuba from tests.
Reviewed By: krallin
Differential Revision: D27449808
fbshipit-source-id: 9c79e83fbfdf3d9a02c2cfc8b6a8255edb4241fe
Summary:
This is going to enable the background update in SegmentedChangelog to log
entries to Scuba.
The scuba sample builder is not fundamentally different than other elements of
the environment. It is used slightly differently to, for example, Logger,
because it has to cloned in all places that want to log rows but otherwise it
has the same characteristics.
Reviewed By: krallin
Differential Revision: D28210008
fbshipit-source-id: 68468868d13f29dddf21095bd7526cb4ff690786
Summary:
This is where async requests are logged to be processed, and from where they
are polled later.
It will acquire more functionality when the actual request processing business
logic is implemented.
Reviewed By: StanislavGlebik
Differential Revision: D28092910
fbshipit-source-id: 00e45229aa2db73fa0ae5a1cf99b8f2d3a162006
Summary:
In very hot code path, EdenFS is spending a very large amount of time creating
and destroying folly::Future objects. This is due to the required memory
allocation, as well as the handful of atomics that are happening at creation
time, and these are showing up in EdenFS profiles.
In the steady state, EdenFS actually doesn't need futures, as it often times is
able to service requests from its in-memory caches, in which case we should
ideally just return the value itself and not wrap it in a folly::Future. The
added ImmediateFuture is a step in this direction, as it can hold either an
immediate value, or a folly::SemiFuture, and allow the same API to be used
transparently between these 2.
Reviewed By: genevievehelsel
Differential Revision: D28006802
fbshipit-source-id: 89eaa32e7fa82c44844c4b23c4cb30dbeea46ca8
Summary:
Log the sizing metadata about keys that scrub has seen to the pack info logs.
This uses the sampling blobstore to see all blobstore gets and captures info from them.
Also updates relatedness_key fieldname to mtime as that way its less easily confused with similarity_key
Differential Revision: D28115620
fbshipit-source-id: 666a444c2d91b0ca5bb225cea971f9b183e6a48d
Summary:
Pass BlobstoreGetData to the sampler so that it has a chance to sample the BlobstoreMetadata as well as the BlobstoreBytes.
This is used in the next diff for sampling the sizing information.
Reviewed By: markbt
Differential Revision: D28115619
fbshipit-source-id: 7a79d482c9ba1ed8b08afab5f1c1b8fe7c4f257a
Summary: When reading from packblob we'd like to see metadata about sizes so that can log it for the packer later in this stack.
Reviewed By: markbt
Differential Revision: D28101971
fbshipit-source-id: 96dd0d5497c2bb5c27231709dbd19d00168e1a77
