Summary:
This introduces some basic unit tests to ensure correctness of the cache.
We are adding tests to cover the simple methods of the object cache since we
are using that code path here. And adding a few sanity check tests to make sure
the cache works with trees.
Reviewed By: chadaustin
Differential Revision: D27050296
fbshipit-source-id: b5f0577c1662483f732bb962c5b40bca8e1dcb40
Summary:
Chad first noted that deserializing trees from the local store can be expensive.
From the thrift side EdenFS does not have a copy of trees in memory. This
means for glob files each of the trees that have not been materialized will be
read from the local store. Since reading an deserializing trees from the local
store can be expensive lets add an in memory cache so that some of these
reads can be satisfied from here instead.
This introduces the class for the in memory cache and is based on the existing
BlobCache. note that we keep the minimum number of entries functionality from
the blob cache. This is unlikely to be needed as trees are much less likely
than blobs to exceed a reasonable cache size limit, but kept since we already
have it.
Reviewed By: chadaustin
Differential Revision: D27050285
fbshipit-source-id: 9dd46419761d32387b6f55ff508b60105edae3af
Summary:
On all the code paths that matter we always acquire the write lock. Since we are
basically just using a simple lock, distributed mutex is a more efficient implementation
that does fancy tricks with cachelines. This improves performance from testing
globs which cause many concurrent reads from the cache.
Reviewed By: chadaustin
Differential Revision: D27810990
fbshipit-source-id: d22470f3f39e2cd3895f5ea772955b62030d154a
Summary:
Now that Object Cache actually does most the work, I am moving the
BlobCache tests to be ObjectCache tests. I am leaving a few blob cache
tests to sanity check that the cache works with blobs.
Reviewed By: chadaustin
Differential Revision: D27776113
fbshipit-source-id: ef58279d93035588beb162ee19173a42e3ca4e5b
Summary:
We would like to use a limited size LRU cache fore trees as well as blobs,
so I am templatizing this to allow us to use this cache for trees.
Trees will not need to use Interest handles, but in the future we could use
this cache for blob metadata, which might want to use interest handles.
Additionally if we at somepoint change the inode tree scheme that would remove
the tree content from the inodes itself, interest handle might be useful for
trees. We could also use this cache proxy hashes which may or may not use
interest handles. Since some caches may want interest handles and others will
not I am creating get/insert functions that work with and without interest
handles.
Reviewed By: chadaustin
Differential Revision: D27797025
fbshipit-source-id: 6db3e6ade56a9f65f851c01eeea5de734371d8f0
Summary:
Thrift setter API is deprecated since it doesn't bring any value over direct assignment. Removing it can reduce build-time and make our codebase more consistent.
If result of `s.set_foo(bar)` is unused, this diff replaces
s.set_foo(bar);
with
s.foo_ref() = bar;
Otherwise, it replaces
s.set_foo(bar)
with
s.foo_ref().emplace(bar)
Reviewed By: xavierd
Differential Revision: D27986185
fbshipit-source-id: d90aaf27f25f2ecfcbbbe7886e0c0d784f607a87
Summary:
This and following diff will refactor CheckoutPlan creation.
Right now we create CheckoutPlan from manifest diff and then manipulate it with methods like `with_sparse_profile` to adjust plan for different cases.
Those 'adjustment' do not work great with the structure of CheckoutPlan, for example `with_sparse_profile` has to create temporary HashSet just to index files in CheckoutPlan
We are going to add more adjustments in the future (for example, checkout --clean), and will run into same issues with current structure of CheckoutPlan
To avoid those issues, we are going to refactor this code, so that instead of Diff -> CheckoutPlan -> adjustments, we are going to have Diff -> ActionMap -> adjustments -> CheckoutPlan
The structure of CheckoutPlan is still good for it's direct purpose (execution), but all the 'changes' to the plan will be done in ActionMap instead.
Reviewed By: DurhamG
Differential Revision: D27980390
fbshipit-source-id: 403f371fd2fe7760984925a38429e1bfb88d8e3f
Summary: For now this does not work with --clean flag(fallback to regular checkout in that case)
Reviewed By: quark-zju
Differential Revision: D27953967
fbshipit-source-id: 71c097cf1e395ff2cba2f4ee528145d3b2c83c23
Summary: This crate does not calculate status, but allows to convert python status into rust status, that can later be used by python code
Reviewed By: quark-zju
Differential Revision: D27953962
fbshipit-source-id: ab91d9d035140e43d8b17988b24bd030af77c96d
Summary: When checking out on dirty copy without --clean this function can be used to check if checkout operation conflicts with currently modified files
Reviewed By: quark-zju
Differential Revision: D27953965
fbshipit-source-id: 4096506e4cbf8b102e0afa1a929c066dfa474825
Summary:
This crate introduces consumer API for status in rust
Currently the implementation will just take status from Python and convert it into this struct
But in the future we can get pure Rust implementation to get status
Reviewed By: quark-zju
Differential Revision: D27953963
fbshipit-source-id: 29c876400c82056eaf81fffa4adc814473853c1e
Summary: This method can be used to 'normalize' path for case insentive use cases
Reviewed By: quark-zju
Differential Revision: D27953964
fbshipit-source-id: 421832af22af9a3b56eec0d045b9f983592ed192
Summary:
Fix missing stats for the bookmarks endpoint because we have the wrong name in
code.
Reviewed By: quark-zju
Differential Revision: D28008091
fbshipit-source-id: 128fe00e00a06ebe9b65fb11512cd03a042d55fe
Summary:
This doesn't link the errors into Scuba, which makes it not very useful for
debugging, since we're not routinely looking at stderr on our tasks, and makes
it impossible to e.g. look anywhere and find a count of failed requests.
Instead, update this to use `capture_first_err`, which will report both the
first error and the total error count to Scuba.
Reviewed By: sfilipco
Differential Revision: D27998037
fbshipit-source-id: b941d44a2ac21bbf640b9bc977de749207f12d9a
Summary:
In EdenAPI, most endpoints don't raise errors when they fail, and instead just take items out of the response stream (though there are some exceptions: D24315399 (0ccae1cef9).
Right now, this just gets logged to stderr, which isn't great. This diff introduces a CaptureFirstError wrapper we can use to capture those errors and expose them in post-response callbacks (i.e. Scuba).
Reviewed By: sfilipco
Differential Revision: D27998036
fbshipit-source-id: 960d0e09a82ca79adfafe22e2eeef2e0294d27dc
Summary:
`lld` is faster than `ld.gold`.
To compare, `make local3`, then add a blank line in `hgmain`, then run `make
local3` again.
Using `lld`:
building 'hgmain' binary
Compiling hgmain v0.1.0 (/home/quark/fbsource-lazy/fbcode/eden/scm/exec/hgmain)
Finished release [optimized + debuginfo] target(s) in 3.73s
building 'mkscratch' binary
Finished release [optimized] target(s) in 0.37s
building 'scm_daemon' binary
Finished release [optimized] target(s) in 0.42s
Using `ld.gold`:
building 'hgmain' binary
Compiling hgmain v0.1.0 (/home/quark/fbsource-lazy/fbcode/eden/scm/exec/hgmain)
Finished release [optimized + debuginfo] target(s) in 10.15s
building 'mkscratch' binary
Finished release [optimized] target(s) in 0.39s
building 'scm_daemon' binary
Finished release [optimized] target(s) in 0.46s
Reviewed By: ikostia
Differential Revision: D28006551
fbshipit-source-id: 55b19be93d8152634d79ed92c9cd53237f91b820
Summary:
This raw XDB connections API is not used in Mononoke although there are projects depending on it.
The API creates connection objects based on shed/sql crate.
This diff moves `SqlConnections` into `shed/sql:sql_common`, closer to the `Connection` definition, and moves `create_raw_xdb_connections` API into the `shed/sql:sql_facebook::raw`.
Reviewed By: krallin
Differential Revision: D28003638
fbshipit-source-id: ea4a29b6e239a89c97237877e2dfde4c7c7ff89b
Summary:
It turns out during the initial clone, we're not using selectivepull for some
tiers that do not have the non-repo selectivepull config.
We've been using selectivepull for devservers and corp (and it's effective
during clone) for a long time. Let's just enable it by default so even if
dynamicconfig does not set it properly, we can still use selectivepull clone
to avoid pulling 10k+ remote bookmarks (which triggers auto bookmark cleanups
as logged in hgfeatures).
There are too many incompatible tests so I'm not migrating them for now.
Reviewed By: DurhamG
Differential Revision: D28006488
fbshipit-source-id: f0dc000156abde530fd8881bd26b4642a36167be
Summary:
As I split the pushrebase crate in D27968029 (93b8cf116b) it makes sense to stop re-exporting hook definitions from it.
This change will also make dependencies more accurate.
Reviewed By: krallin
Differential Revision: D28028045
fbshipit-source-id: 622ef5d7db737e19153109f4d2dcefe54fba2bb4
Summary:
Finally! This is basically the end goal of this stack (though we still need to
do the same thing with the "ForwardErr" combinator used by EdenAPI next).
With this, we can now capture errors that occur while sending the response
stream, instead of just errors that occur while producing the response (which
is usually a little piece of the work we have to do).
Reviewed By: mitrandir77
Differential Revision: D27967991
fbshipit-source-id: a5483c58f0550a19e711e712cf860d9328a0eb9e
Summary:
`ContentMeta` sounds a lot like a struct containing content metadata, so let's
rename this to something less ambiguous (`ContentMetaProvider`).
The reaosn I'm doing this now is because I'd like to introduce a similar
trait for errors that occur on our response streams and there I'll need
both a struct and a trait.
Reviewed By: mitrandir77
Differential Revision: D27998039
fbshipit-source-id: f0372c62d13167ef4bd07cb9e9e9fb75ea105b9a
Summary:
Like it says in the title. The goal here is to make it not matter where the
error came from. In this diff, we capture the same errors as before, but we
do it via PostResponseInfo instead of via ad-hoc things in our `State`.
Reviewed By: mitrandir77
Differential Revision: D27967994
fbshipit-source-id: dbbb1a8f5ea1a439089c41b4a08cd6088476ae33
Summary: Like it says in the title.
Reviewed By: mitrandir77
Differential Revision: D27967992
fbshipit-source-id: 0deb4d90538a6889bee6b41de4c5d1533b29519b
Summary:
Very small refactor. I want this stuff to all be in the same module instead
of spread across `response` and `error`.
Reviewed By: mitrandir77
Differential Revision: D27967993
fbshipit-source-id: aca22f952d756d298e5e342f0c4f8ebd31f108bf
Summary:
This is a bit of an abstract change, but the goal of this change is to make
post-response handlers oblivious to the distinction between sending a response
(or failing to send one) and returning a response that actually contains a
(fallible) stream.
The underlying / ultimate goal here is to move our error reporting out of
ad-hoc router wrappers where we call `set_error_message` on some context
entity, and to instead move it into post-response callbacks.
The upshot of that is that if we fail to send a response even though we sent a
200 from the handler, we'll be able to log it! Indeed, remember that when
sending a streaming response, we have to send a 200 to start streaming but we
might hit an error producing later parts of the response!
Reviewed By: mitrandir77
Differential Revision: D27966422
fbshipit-source-id: ab49639bfcc4af23ddc2e84181278f105ebb28b9
Summary:
This stuff runs after we sent the response, so PostResponse is a more
appropriate name than PostRequest.
Reviewed By: ikostia
Differential Revision: D27966420
fbshipit-source-id: 1f7b7a55490f731eb512793024bcfafb0ea4ef79
Summary:
Those two have historically used different (but largely copy pasted) code to
produce their responses. Let's unify them by
While in there, let's also modernize the formatting a little bit by letting
anyhow do the formatting for us (we used to use `failure` when this code was
written, and it didn't do it).
There's a bit of ugliness here in the sense that out formatting is injecting
the error into the state so it can be logger later. This is equivalent to what
we had, but it's kinda clonwy. That said, I'm working on refactoring our error
handling in this stack, so the plan is to clean this up (i.e. it won't stay
very long).
Finally, note that this diff removes the `ResponseCreationFailure` error kind
in LFS. This is replaced by a `.context()` in `gotham_ext`. That said, we never
really use this stuff: creations are fallible in Hyper but you only run into
an error if you e.g. forget to add a status code, so we don't expect them to
actually occur outside of development.
Reviewed By: mitrandir77
Differential Revision: D27966421
fbshipit-source-id: 097f3b69f25fe39f8fbef925a272e88199896b39
Summary:
Like it says in the title, I'd like to use names that reflect that this isn't
just *any* content stream: it's specifically for responses.
Reviewed By: ahornby
Differential Revision: D27964045
fbshipit-source-id: 50530cf85ba7840a2baa14151351d0b288d9ae70
Summary:
We have a set of things that are meant to be used together that are spread
across 3-4 different modules. Let's move them together. This also allows us to
make some things (e.g. the `ContentMeta` trait) private as they're no longer
needed.
Note: this diff only move stuff around & renames things a bit. I'll also split
some of those modules in the next diff.
Reviewed By: HarveyHunt
Differential Revision: D27964047
fbshipit-source-id: 02528d84adfd70ec346c32163cb185d89266a53e
Summary:
We have a module called "content" that contains two completely unrelated
things: some enums for content encodings (+ associated parsing), and our output
streams.
I'd like to add more of said output streams, so let's clean this up.
Reviewed By: HarveyHunt
Differential Revision: D27964046
fbshipit-source-id: b42e56aa3fadaf9b93a44216977da19d950a16b9
Summary:
We used to have to do this because of overly strict trait bounds in Hyper, but
we no longer do, so let's get rid of it. Note that we have one Tokio task per
request, and polling the response stream is basically the only thing that task
does, so this should make little difference as far as task scheduling is
concerned besides avoiding unnecessary context switches.
Reviewed By: ahornby
Differential Revision: D27963458
fbshipit-source-id: 24e762eb173156dab909fefe11dcf2d58272048a
Summary: We don't use this anymore. Might as well remove it.
Reviewed By: ahornby
Differential Revision: D27963411
fbshipit-source-id: a6ac337936e8b2bd788dd79a835eef11b19dde70
Summary: It has been fixed and we now set auth config with higher priority anyway.
Reviewed By: johansglock
Differential Revision: D28026081
fbshipit-source-id: 7086b48139bb05ffadd782898a1758ae06236aca
Summary:
This change makes it so that our binaries do not instantiate a real configo
client in integration test setup.
Reviewed By: ahornby
Differential Revision: D28026790
fbshipit-source-id: 0fb9ce66a1324e845f4b8a80d4479263ec6e4ee1
Summary:
First, some background on the existing WrappedPath type: In Mononoke the MPath type is such that None==Root and Some(MPath)==NonRoot. This means that where a path may be present one needs to use double-Option with Option<Option<MPath>>, so that Root is Some(None).
To reduce the need for double Option, and subsequently to allow for newtype features like memoization, the walker has WrappedPath, so we can use Option<WrappedPath> instead.
This change introduces a similar type WrappedPathHash for MPathHash, which means that the sample_fingerprint for WrappedPath can be now be non-optional as even root paths/manifests can now have a sample_fingerprint.
Reviewed By: mitrandir77
Differential Revision: D27995143
fbshipit-source-id: b674abd4ec94749f4f5797c697ae7381e1a08d02
Summary:
This adds the first part of new logging from the walker that can be used to gather details on what keys might make sense to pack together.
Unlike the corpus command that dumps file content by path (which was useful for analysis on compression approaches), this adds logging to the scrub command that includes the path hash rather than the full path. This should keep memory usage down during the run, hopefully mean we log from existing scrub jobs and and mean the logs are more compact.
Reviewed By: mitrandir77
Differential Revision: D27974448
fbshipit-source-id: 47b55112b47e9b022f16fbb473cf233a7d46bcf3
Summary:
Knowing the prepushrebase changeset id is required for retroactive_review.
retroactive_review checks landed commits, but verify_integrity hook runs on a commit before landing. This way the landed commit has no straightforward connection with the original one and retroactive_review can't acknowledge if verify_integrity have seen it.
Reviewed By: krallin
Differential Revision: D27911317
fbshipit-source-id: f7bb0cfbd54fa6ad2ed27fb9d4d67b9f087879f1
Summary:
Split pushrebase crate into pushrebase hook definition and pushrebase implementation.
Before this change it was impossible to store an attribute in BlobRepo that would depend on PushrebaseHook as it would create a circular dependency `pushrebase -> blobrepo -> pushrebase`.
Reviewed By: krallin
Differential Revision: D27968029
fbshipit-source-id: 030ef1b02216546cd3658de1f417bc52b8dd9843
Summary:
It's rare (only seem to be used by chistedit) but if revrange got an int, the
expected behavior is to treat it as a revision number.
Reviewed By: kulshrax
Differential Revision: D27983989
fbshipit-source-id: f9f8d9cb39af4ec1de7ed8ca69f7f1879b4a4614
Summary:
Adds a message for users to use 'hg checkout --continue' if there's a
.hg/updatestate file (indicating an aborted checkout) and if they're on the null
rev (indicating they likely just cloned).
Also adds support for 'hg checkout --continue' to work with non-merge commits.
Note, it really only currently works when checking out from null, since
otherwise there will be a lot of modified files in the way. Once native checkout
is more mature, we can teach it to ignore modified files that match the desired
checkout destination.
Reviewed By: quark-zju
Differential Revision: D26967976
fbshipit-source-id: 7397c5fe82027e22bf1b4db0f14bb180239fae25
Summary:
The check unknown logic would block checkout for any unknown files that
were to be overwritten. We want to allow checkouts where the unknown file has
the same content as the desired checkout value. Ideally we'd check it against
the SHA1 hash of the file we're about to checkout, but since content hashes
aren't available yet we can limit our check to resumed checkouts for now.
Reviewed By: andll
Differential Revision: D27804719
fbshipit-source-id: e129ca694080051420e2cb685c7eeb5f1adee005
Summary:
Every function on CheckoutPlan required the VFS already, and the
CheckoutProgress is storing the VFS and living on the CheckoutPlan, so it makes
sense to just store the VFS on the CheckoutPlan.
Reviewed By: andll
Differential Revision: D27825088
fbshipit-source-id: 3d063fdfd1a50983b60d00a3992a893e71732f94
Summary:
Now that CheckoutPlan can look for untracked files, it breaks the
ability to continue a checkout since those untracked files are considered dirty.
In a later diff we'll use the CheckoutProgress to inspect the dirty files and
determine which are actually dirty and which can be overwritten. To do so
though, we need access to the CheckoutProgress earlier. So let's just store it
on the CheckoutPlan.
This is a little awkward because we're passing the root VFS to the constructor
so CheckoutProgress can be instantiated, but then also passing it to every
CheckoutPlan function as well. We should probably just store the vfs on the
CheckoutPlan. If others agree, I can make a diff to do that.
Reviewed By: andll
Differential Revision: D27804720
fbshipit-source-id: e819c27fa8580c82a8cf8f0baf22ac1ea707ee54
Summary:
Python on Windows has a couple bugs around passing stdin/out/err to
subprocesses. In Python 2 we patched our Python to fix one of the bugs. With
Python 3 we're using the vanilla Python distribution, so we'd rather not patch
it.
Instead, let's fix it by passing stdin/out/err explicitly.
See D15764537 (b8c747b6d5) for the details of the bug.
Reviewed By: kulshrax
Differential Revision: D28010523
fbshipit-source-id: d88f789a8100b04da996271de7dfe566c0f715df
Summary: Allows us to background a prefetch (similar to how prefetch-profile fetches are backgrounded). A thing to note here is that we do not deduplicate fetches for prefetches, but if there is enough busy work between bulk filesystem accesses and the prefetch finishing, this should not be an issue.
Reviewed By: chadaustin
Differential Revision: D27028428
fbshipit-source-id: 5c528fff76719f42151542eaa3499271f7ab6fa3
Summary:
This diff wires up actual `scs_server` methods `megarepo_add_sync_target_config`, `megarepo_add_sync_target`, `megarepo_add_sync_target_poll` to the underlying logic in the `CfgrMononokeMegarepoConfigs`.
One of these is a synchronous method (`megarepo_add_sync_target_config`), so it is implemented properly. This method only allows adding new configs to an existing target.
The other two are a pair of async methods (create reqest + poll request) for target creation with an initial config. On the one hand, we don't yet have any infrastructure for async methods, so properly implementing this pair is not possible. What's more, target creation is a two-part operation: save a config + create an initial repo commit. Second part is not yet implemented at all (and is what requires async implementation, as it is going to be expensive). On the other hand, I would like to expose the concept of target creation for the client to test, that's why I add `FAKE_ADD_TARGET_TOKEN` to mask a so-far synchronous impl of this method as asynchronous.
Once I implement async methods, I will come back and work on a proper `megarepo_add_sync_target` impl (this is the first method to be implemented).
Important: any use of these methods now should be considered experimental, and we'll have to delete all of these configs later (because all of the targets won't have any corresponding bookmarks in the real repos, which makes them invalid).
Reviewed By: StanislavGlebik
Differential Revision: D27885979
fbshipit-source-id: 9e2a914af1a7db2ec00ffa11a832ddd71fd19d0f
Summary:
This will help people to introduce new blobstore objects in their code (for
instance I intend to use it in the following diff).
The `private` module exists to allow the use of the exported macro without the
need to write a bunch of `use` statements, and without pollution the re-export
namespace. The idea is that everything needed by the exported macro exists in
the `private` module of the crate, and this module is public.
So long as some other crate imports the macro, it expands to
`$crate::private::NEEDED_THING` in the right places and no further `use`
statements of dependencies are needed. At the same time, the name `private`
should discourage people from using whatever is in this module directly. The
idea is taken from `anyhow`.
Reviewed By: StanislavGlebik
Differential Revision: D27997228
fbshipit-source-id: fd2c421d0daf0fe88e2b9001bb4088fe7b4d59b7
Summary:
Collecting into SortedVecMap an unsorted iterator is inefficient, because of
how [try_collect()](https://docs.rs/futures-util/0.3.0/src/futures_util/stream/try_stream/try_collect.rs.html#57) works. It calls `extend()` method every time a new element was fetched from the
stream. So if stream was unsorted, then we ran into a quadratic behaviour with
lots of reallocations.
Let's instead collect into BTreeMap, and then convert it into SortedVecMap.
Reviewed By: markbt
Differential Revision: D27997469
fbshipit-source-id: 58f837e6cc946ccc8809cce3d7a5a2e6ca24df40
Summary:
I'd like to have a quick way of documenting how this is supposed to be used,
so let's add it.
Reviewed By: HarveyHunt
Differential Revision: D27996500
fbshipit-source-id: 0d138ac3335a3ecb7f0e15aebbdc89d01941cbed
Summary: In python 3 these strings are already unicode, so let's just .upper() them. Otherwise it crashes with 'no decode() on str'. This only impacts eden checkouts, since non-eden uses treestate which doesn't use this codepath.
Reviewed By: quark-zju
Differential Revision: D27978369
fbshipit-source-id: a298c1b455fdb8aa09db0ac667bd97b8e419bbe8
Summary:
During pull, commitcloud may try to auto join a cloud workspace. If
there are no certs, the join will fail and will cause the overall pull to exit
non-zero. Let's just print a warning instead and allow the pull to succeed.
Reviewed By: sfilipco
Differential Revision: D27928397
fbshipit-source-id: 432ee589438bb5af9f47f7aaa735bbbb5a17ad6b
Summary:
These methods will be used in my later Windows fsck diff as it will need to scan disk state to find changes.
It is a bit unfortunate that we'll need to stick with boost for now. However this should be a fairly easy migration to `std::filesystem` once that is available.
Reviewed By: kmancini
Differential Revision: D27872828
fbshipit-source-id: f6b27a171026aeaaea3db9f17b8f43cfa25004e4
Summary:
Currently we have limited test coverage of the NFS code. Let's start running
our integration tests on NFS mounts. We already duplicate tests to run them on
both Git and Hg repos using a python decorator. We can update this decorator to
run a copy of tests on an nfs mount.
This covers most of the tests, but a few tests do not use this decorator. See next
change.
Note some tests are currently broken, so I am using the same skip list functionality
we use for windows so we use a uniform framework.
Reviewed By: xavierd
Differential Revision: D27874662
fbshipit-source-id: c7d425830b691e395b5228d0e0f797f67987b4ec
Summary:
Copy from Watchman.
This allows us to show stack trace when EdenFS terminates on Windows.
Reviewed By: chadaustin
Differential Revision: D27896966
fbshipit-source-id: f3238a37a1176f879d5e6bc051ec97031c9a7096
Summary:
Add a way to extend the graph with concrete commit hashes, without specifying
exact commit messages.
Reviewed By: sfilipco
Differential Revision: D27897894
fbshipit-source-id: fccd64b2fef1386d79cddd841208da6a938a5217
Summary:
scrub blobstore logging was missing the common server logging fields that LogBlob and MultiplexedBlobstore add.
Also moved the LogBlob scuba construction closer to use site for clarity.
Reviewed By: StanislavGlebik
Differential Revision: D27966453
fbshipit-source-id: 77fe70606602753301a2503691a490c0b11c755a
Summary:
Currently when we call with_mutated_scuba() we create a new LoggingContainer.
That means that all the data from the previous LoggingContainer like PerfCounters
is lost. I suspect this is the reason we don't log any BlobGets/Puts for
repo_create_commit methods (see
[scuba](https://fburl.com/scuba/mononoke_scs_server/fautos3s)) - we call
[with_mutated_scuba method](https://fburl.com/code/srd1c4xu) right before
calling repo_create_commit(), and I suspect this loses the counters.
Let's instead copy all the Logging fields when calling `with_mutated_scuba`.
Reviewed By: krallin
Differential Revision: D27964719
fbshipit-source-id: 881c11bb5fb1927dbf55d0d625ea8bfbf11be131
Summary:
In test-cross-repo-commit-validator.t and test-cross-repo-commit-sync., we
modify bookmarks outside of Mononoke, so we need to flush them before pulling.
In test-megarepo-invisible-merge.t, things are actually a little more subtle
and I wonder if there might be another issue laying around there. If we don't
flush bookmarks, then we attempt to upload one more hg commit, and that
blows up: P410235472. However, if we flush bookmarks, then we don't attempt to
upload, and all is fine. Here, flushing is just a workaround, but for no
that'll do. There was also another bug here, where we change configs but
don't force them to take effect.
Reviewed By: StanislavGlebik
Differential Revision: D27964959
fbshipit-source-id: 9c4304b38513177e402ee64f309e019e227ed2a7
Summary:
When we're packing, we pay an overhead price for keeping the key in the pack. As we're only bothered about reducing size, let's limit that price to when the savings from packing are worth it.
There are two cases where it's not worth it:
1. When the compressed pack is larger than the sum of single compressed files sizes.
2. When compressing a single file on its own.
Reviewed By: ahornby
Differential Revision: D27913258
fbshipit-source-id: 36cdc2a2b30aa508281ac3bbd70da41322533edb
Summary:
Like it says in the title. This test wants to see consistent bookmarks so it
should flush them.
Note that this used to just opt out of bookmarks caching entirely, but I'd like
us to try and avoid having so many snowflakes in our tests (because it makes
their maintenance harder), so instead of changing the environment, let's change
the test to do what it needs to do.
Reviewed By: mzr, HarveyHunt
Differential Revision: D27964099
fbshipit-source-id: 72e00bad07dec15f18faaf4aa2e32e78cb333ab0
Summary: Was getting orphan autocargo lints on these so add a config for them.
Reviewed By: krallin
Differential Revision: D27947231
fbshipit-source-id: 925fb78889d8f80f51145536a157fa0e63cc68d7
Summary:
Current implementation had a bug(demonstrated in test case) in handling unknown files on case insensitive fs.
When file is replaced with another file, whose name only differs in case, we get two distinct update operations - rm file, and create file.
Create operation checks against unknown files, and see that file "exists". In this case operation is aborted.
However, we should proceed in this case, and this diff fixes it.
Reviewed By: quark-zju
Differential Revision: D27926953
fbshipit-source-id: 48c8824322d6e5dd9ae57fee1f849b57dc11a4df
Summary: Will be useful on case insensitive fs
Reviewed By: quark-zju
Differential Revision: D27946982
fbshipit-source-id: e7a2fd0ee503c4a580531e6f52225fe2316e5b76
Summary: This diff adds flag to VFS to detect whether FS is case sensitive. The logic in this code losely follows similar logic in Python
Reviewed By: quark-zju
Differential Revision: D27926952
fbshipit-source-id: 36fdf4187ae513b25346f704050c64f9a1a4ec74
Summary: This way the fallback server know which traffic is coming from mononoke
Reviewed By: krallin
Differential Revision: D27946019
fbshipit-source-id: 8c13ae641ba340ba55322871ca30fb6accb3f007
Summary:
Update the zstd crates.
This also patches async-compression crate to point at my fork until upstream PR https://github.com/Nemo157/async-compression/pull/117 to update to zstd 1.4.9 can land.
Reviewed By: jsgf, dtolnay
Differential Revision: D27942174
fbshipit-source-id: 26e604d71417e6910a02ec27142c3a16ea516c2b
Summary:
When EdenFS is killed, either due to `eden stop` timing out, or when simply
rebooting the host, the edenfs.log becomes filled with fsck errors, which also
slows down the fsck process.
Since we already print the number of errors per mount, limiting ourself to the
first 50 errors is probably good enough.
Reviewed By: fanzeyi
Differential Revision: D27943618
fbshipit-source-id: 2b3e6e3ae4df648d4b1dccf73c71f8dbbded3892
Summary:
We get pretty frequent query errors from MySQL on this, but it's hard to debug
without knowing what is being queried.
Reviewed By: StanislavGlebik
Differential Revision: D27941603
fbshipit-source-id: 62e0f0fe9c3af36ed829c401e957ecf7683a4000
Summary: Mercurial still needs this to work in Python 2 for a few more weeks.
Reviewed By: quark-zju, xavierd
Differential Revision: D27943521
fbshipit-source-id: 2b5106496fbb523cdc97a3dce3ad0cbfab5c17b7
Summary:
This handles large chunk of cases where tree merge returns conflict, but the conflict can be trivialy resolved by textual merge.
No markers are left in file, if merge yields conflicts we simple abort to on-disk merge, same as with existing code
Reviewed By: quark-zju
Differential Revision: D27752771
fbshipit-source-id: ff8d4bbc88b48812150327cae6e31991a30236c9
Summary: Those conflicts can be resolved in Python using textual 3-way merge
Reviewed By: DurhamG
Differential Revision: D27752770
fbshipit-source-id: 816a601112ee2e747d780f8b17473049df46b469
Summary:
This diff modifies rebase flow(based on config) and attempts to create commit wihthout using merge.py:update
This currently passes some test cases, but not all.
This implementaiton currently does not attempt to resolve conflicts and fallbacks to on-disk merge if they are encountered.
This fails some test cases, because they expect some trivial conflicts to be resolved by in-memory merge.
There are also certain rebase flags that currently are not handled
Reviewed By: DurhamG
Differential Revision: D27639394
fbshipit-source-id: d8f71e955930e3a8a64d7d95a0cf184d9b4ccadc
Summary: This diff exposes manifestbuilder that can be used to construct memctx from Python
Reviewed By: DurhamG
Differential Revision: D27639395
fbshipit-source-id: ed047d3d7533f9d2bc592a5d948dc01e429692a7
Summary:
This diff makes MySQL FFI client the default option for a MySQL connection. It means that if no arguments provided, the MySQL FFI client is used. `--use-mysql-client` option is still accepted, as it is used in the configs, and will be removed a bit later.
I also removed raw connections as a way to connect to MySQL from Mononoke, as it is no longer used. Although I had to keep some `sql_ext` API for now because other projects rely on it.
(I talked to the teams and they are willing to switch to the new client as well. I'm helping where it's possible to replace these raw xdb conns.)
Reviewed By: krallin
Differential Revision: D27925435
fbshipit-source-id: 4f08eef07df676a4e6be58b6e351be3e3d3e8ab7
Summary:
Right now, we can't have defaults in our tunables, because some tests clobber
them. Let's start updating tunables instead of replacing them.
NOTE: I was planning to use this in my next diff, but I ended up not needing
it, that said, it feels like a generally positive improvements, so I figured
I'd keep it.
Reviewed By: StanislavGlebik
Differential Revision: D27915402
fbshipit-source-id: feeb868d99565a375e4e9352520f05493be94a63
Summary:
This updates the bookmarks cache TTL to be something we configure using
tunables instead of repo configs. There's a few goals here:
- Less us tune the pressure we put on SQL via a tunable
- Letting us experiment more easily with disabling this cache and tuning the
WBC poll interval.
Right now, this defaults to what we had set up in all our repo configs, which
is 2000ms.
Reviewed By: farnz
Differential Revision: D27915403
fbshipit-source-id: 4361d38939e5b2a0fe37442ed8c1dd0611af5098
Summary:
One of our plans for this half is to replace the warm bookmarks cache with a
service, and we suspect this will effectively eliminate bookmarks queries from
our hosts, because we think they all come from the WBC.
But, before we invest our time into this, let's make sure that this assumption
is actually correct, by tracking who's querying bookmarks.
Reviewed By: StanislavGlebik
Differential Revision: D27938407
fbshipit-source-id: d9a9298e7409c9518a4b9bf8ac0a6cef53750473
Summary:
I'd like to be able to track the proportion of traffic coming to bookmarks from
warm bookmarks cache vs. from elsewhere. We don't have a great abstraction to
pass this via the CoreContext at this time, but the SessionClass seems like a
pretty good fit.
Indeed, since it's always available in the CoreContext, and can be freely
mutated without having to rebuild the whole session. Besides, it aligns pretty
well with the existing use cases we have for SessionClass, which is to give you
different level of service depending on who you are.
Reviewed By: StanislavGlebik
Differential Revision: D27938413
fbshipit-source-id: a9dcc5a10c8d1459ee9586324a727c668e2e4e40
Summary:
phases calculation could be expensive on the server and it should be a perf win to disable it if not needed
It shouldn't be needed if narrow heads are enabled
Reviewed By: quark-zju
Differential Revision: D27908691
fbshipit-source-id: 7000fb23f9332d58c2c488ffbef14d73af4ac532
Summary:
`MononokeMegarepoConfig` is going to be a single point of access to
config storage system - provide both writes and reads. It is also a trait, to
allow for unit-test implementations later.
This diff introduces a trait, as well as implements the write side of the
configerator-based implementor. The read side/oss impl/test impl
is left `unimplemented`. Read side and test impl will be implemented in the future.
Things I had to consider while implementing this:
- I wanted to store each version of `SyncTargetConfig` in an individual
`.cconf` in configerator
- at the same time, I did not want all of them to live in the same dir, to
avoid having dirs with thousands of files in it
- dir sharding uses sha1 of the target repo + target bookmark + version name,
then separates it into a dir name and a file name, like git does
- this means that these `.cconf` files are not "human-addressable" in the
configerator repo
- to help this, each new config creation also creates an entry in one of the
"index" files: human-readable maps from target + version name to a
corresponding `.cconf`
- using a single index file is also impractical, so these are separated by
ascification of the repo_id + bookmark name
Note: this design means that there's no automatic way to fetch the list of all
targets in use. This can be bypassed by maintaining an extra index layer, whihc
will list all the targets. I don't think this is very important atm.
Reviewed By: StanislavGlebik
Differential Revision: D27795663
fbshipit-source-id: 4d824ee4320c8be5187915b23e9c9d261c198fe1
Summary:
We started getting the message
```stderr: eden/fs/utils/SpawnedProcess.cpp:798:21: error: 'getIOExecutor' is deprecated: getIOExecutor is deprecated. To use the global mutable executor use getUnsafeMutableGlobalIOExecutor. For a better solution use getGlobalIOExecutor. [-Werror,-Wdeprecated-declarations]
```
I don't see why we would need a mutable executor here so I chose `getGlobalIOExecutor` over `getUnsafeMutableGlobalIOExecutor`.
Reviewed By: kmancini
Differential Revision: D27912276
fbshipit-source-id: 95b1053f72c2b4eb2746e3c40c0cf76b69d90d6e
Summary:
In case the Mononoke server cannot provide the commit graph, and we need to
checkout and push changes. Let's add an emergency mode where the commit graph
only contains a single commit: master.
This can be used using `--config unsafe.emergency-clone=1`:
~/hg % lhg clone --shallow -U mononoke://mononoke.internal.tfbnw.net/fbsource ~/tmp/c1 --config unsafe.emergency-clone=1 --configfile /data/users/quark/.eden-backing-repos/fbs-lazy/.hg/hgrc.dynamic
connected to <remote host> session yyvXqQlHnMYQMEfw
warning: cloning as emergency commit+push use-case only! accessing older commits is broken!
resolving master
connected to <remote host> session ODc4PPiJ21L6r4Sn
added master: 248bd246f4467a2d4d0cacc09c5e55131ada9919
warning: this repo was cloned for emergency commit+push use-case only! accessing older commits is broken!
Smartlog:
~/hg % cd ~/tmp/c1
~/tmp/c1 % lhg sl
warning: this repo was cloned for emergency commit+push use-case only! accessing older commits is broken!
o 248bd246f 25 seconds ago remote/master
Pull:
~/tmp/c1 % lhg pull
warning: this repo was cloned for emergency commit+push use-case only! accessing older commits is broken!
pulling from ssh://hg.vip.facebook.com//data/scm/fbsource?stage1_read
connected to twshared1103.03.prn6.facebook.com session L4sDKzLm093aLUbo
searching for changes
adding commits
adding manifests
adding file changes
added 8 commits with 0 changes to 0 files
Checkout:
~/tmp/c1 % lhg sparse include .gitignore
warning: this repo was cloned for emergency commit+push use-case only! accessing older commits is broken!
~/tmp/c1 % lhg up master
warning: this repo was cloned for emergency commit+push use-case only! accessing older commits is broken!
19 files updated, 0 files merged, 0 files removed, 0 files unresolved
Commit:
~/tmp/c1 % vim .gitignore
~/tmp/c1 % lhg c -m gitignorewarning: this repo was cloned for emergency commit+push use-case only! accessing older commits is broken!
Smartlog:
~/tmp/c1 % lhg sl
warning: this repo was cloned for emergency commit+push use-case only! accessing older commits is broken!
@ cc43f0e5b (Backup pending) 4 seconds ago quark
╭─╯ gitignore
│
o 10ef2879e 5 minutes ago remote/master
│
~
Reviewed By: andll
Differential Revision: D27897892
fbshipit-source-id: f1770482455968dac217c9c6ee34ec0a20e5f432
Summary:
I found that there are still lots of (automation) users use the legacy clone
code path but it's unclear why (not having selectivepull?). Let's log the
reasons why the legacy path is used.
Reviewed By: sfilipco
Differential Revision: D27913616
fbshipit-source-id: b83f15e42a4afa94164b68bc9a91b4f0c022260c
Summary: The config is True everywhere for a long time.
Reviewed By: sfilipco
Differential Revision: D27913615
fbshipit-source-id: f2af34323c38f11db6bf3137652adea0bf38e858
Summary:
Non-remotefilelog logic might want to use edenapi too. So let's move it to
core.
It might make sense to make the `None` case an explicit error saying
`edenapi.url` is not set. For now I just keep it for compatibility.
The edenapi module is added so one can construct the edenapi client
without using a repo.
Reviewed By: kulshrax
Differential Revision: D27897889
fbshipit-source-id: 2a1fdf4c68464873f294ac1423d2348c1e526d5f
Summary:
Expose the correlator to core. This also reduces the lifetime of correlator
from global (process lifetime) to ui (dispatch.request/command), which
makes more sense and is more compatible with a multi-command per
process world (not using it by default yet).
This is needed to move edenapi to core.
Reviewed By: kulshrax
Differential Revision: D27897891
fbshipit-source-id: 7bd7e422c15e09a82e726436f92d4315ae876d94
Summary:
The zstore for commit messages are now handled by Rust entirely. There is no
need to keep the Python zstore around, except for migration and doctor
use-cases.
Reviewed By: andll
Differential Revision: D27897893
fbshipit-source-id: 21b10596af28c45425f6f102fd13f0421d1e8373
Summary: Add edenapi client python bindings for bookmarks so we use the bookmark api in python mercurial. At this layer there's a mixture of methods that iterate through the response stream and await the stats future and methods that directly return the response stream and stat future. I wasn't certain what to do in this case. The former may be cleaner later on, but the latter may be desirable if we paginate bookmark --list-remote output and therefore don't end up consuming the entire stream.
Reviewed By: kulshrax
Differential Revision: D27331446
fbshipit-source-id: 4dcbcb4bbd69117e02f2028e527f3d6f91c9bf99
Summary: Add bookmark method to Edenapi client for use in Mercurial client.
Reviewed By: kulshrax
Differential Revision: D27174441
fbshipit-source-id: cdc324e9115e87eab2e078209bbbc266e4e1dcdc
Summary:
The `shallowclone` method uses (legacy) revlog. We'll add more APIs that clone
in different ways but all of them use "shallow" (or remotefilelog). Name it to
clarify.
Reviewed By: andll
Differential Revision: D27897890
fbshipit-source-id: 2397a9621d3b207c394c995dff54deda4016e6fa
Summary:
It is considered an anti-pattern (https://rust-unofficial.github.io/patterns/anti_patterns/deny-warnings.html)
and is causing Github CI breakage unnecessarily (https://github.com/facebookexperimental/eden/runs/2402094456):
error: function is never used: `example_blob`
--> src/lfs.rs:1860:8
|
1860 | fn example_blob() -> (Sha256, usize, Bytes) {
| ^^^^^^^^^^^^
|
note: the lint level is defined here
--> src/lib.rs:125:9
|
125 | #![deny(warnings)]
| ^^^^^^^^
= note: `#[deny(dead_code)]` implied by `#[deny(warnings)]`
Reviewed By: andll
Differential Revision: D27911477
fbshipit-source-id: df8d642fe74fe311eb0f329d977b9b8270c27bf4
Summary:
Like it says in the title. It would be a good idea for us to have support for
tweaking this so that we can control the load it forces onto our backend
storage.
Reviewed By: StanislavGlebik
Differential Revision: D27909638
fbshipit-source-id: 1783e1fddbbd609877ff3a9b3084f12e005fca4b
Summary:
Right now, this function has:
- Sleeping
- Spawning
- Doing the work
Let's split this up into a few separate parts to make it easier to refactor.
Concretely I want to make the sleep interval tunable and it's much easier to do
this if that doesn't affect the tests.
Reviewed By: StanislavGlebik
Differential Revision: D27908957
fbshipit-source-id: d28388b8eb8c8f2e8693e1c765d1417bfc4b4825
Summary:
I'd like to make this a tunable, and having some tests turn it off makes that
more complicated. We can now actually flush this, so it also seems a bit
unnecessary. Let's remove.
Reviewed By: StanislavGlebik
Differential Revision: D27911049
fbshipit-source-id: 4634f9f3243132c53c228bd7476002412bace873
Summary:
In case the stderr is redirected but stdout is not:
EDENSCM_LOG=edenscm::mercurial=trace lhg log archival.py 2>/tmp/1
cat /tmp/1
The expected behavior is to still use the pager for stdout output, and the
stderr should be redirected to the specified file.
Reviewed By: andll
Differential Revision: D27867884
fbshipit-source-id: c369bc6be40fc200c4c0e2c9bb38b5faeb1208f2
Summary: This will be used in smartset layer to avoid 1-by-1 fetches.
Reviewed By: andll
Differential Revision: D27867668
fbshipit-source-id: 9d0d30ea214a1176e8c62e25171252c70df707b3
Summary: Added an API to record Python tracebacks to figure out 1-by-1 fetches.
Reviewed By: andll
Differential Revision: D27867670
fbshipit-source-id: d6941dd385db6eb2f810d97815056b660b970032
Summary:
`shorttraceback` returns a shorter version of the "traceback". It will be used
to figure out (frequent) callsite that triggers suboptimal 1-by-1 fetches.
Reviewed By: andll
Differential Revision: D27867673
fbshipit-source-id: 8f1f823007c2c94e1f62e72d816f03cbb4c8bc59
Summary: Split `smarttraceback` so the frame extraction logic can be reused.
Reviewed By: andll
Differential Revision: D27867669
fbshipit-source-id: 0e198f5400df5c1841926f9fac30f70ae74e8108
Summary: Expose Rust APIs to test if a callsite is enabled or not.
Reviewed By: andll
Differential Revision: D27867674
fbshipit-source-id: 0734b5ad6a65040f41a6f8b1bfc1e9a9109b9a8d
Summary:
This will be useful to test if a (frequently called) callsite is enabled or not
without having the logging side effect.
Reviewed By: andll
Differential Revision: D27867672
fbshipit-source-id: 361cb18a7a4680932dcfc9d5496d2906e1dc1f9f
Summary:
The previous choice of the upload / download arrows do not render in Windows
`cmd.exe` terminal using common fonts:
- Cascadia Code
- Cascadia Mono
- Consolas
- Courier New
- JetBrains Mono
- Lucida Console
Replace them with tri-angles that can be rendered using the above fonts.
Note: newer terminals like Microsoft Terminal and WezTerm can render the characters
just fine.
Reviewed By: andll, ikostia
Differential Revision: D27894085
fbshipit-source-id: c53d995355e66ba793d80afc1b36fc83853d07f8
Summary:
Cargo builds run multiple tests in same process, which was causing a failure since MononokeMatches started doing the one time log::set_boxed_logger() call.
I made MononokeMatches:new() error rather than panic in this case as it was already returning a Result anyway.
The failing test was only testing a very small section of code in block_execute, so this change removes it (its covered by integration test). It also gave some MononokeMatches coverage as a side effect (also covered by integration test).
Reviewed By: krallin
Differential Revision: D27891187
fbshipit-source-id: 9787029b610040cbf0125ab79748d6a3e540d2ae
Summary: Remove gotham and hyper patches as referenced PRs have been released
Reviewed By: krallin
Differential Revision: D27905248
fbshipit-source-id: a2b25562614b71b25536b29bb1657a3f3a5de83c
Summary:
We want to distinguish user vs system errors in `configo_client` and its users
(`mononoke_config` for instance). The reason is to allow `scs_server`
distinguish the two types of errors. Normally user errors would only ever be
instantiated fairly "shallowly" in the `scs_server` itself, but `configo_client`
is a transactional client (by this I mean that it calls user-provided transformation functions on fetched data), so we need to allow for a user error to originate from these updater functions.
Reviewed By: StanislavGlebik
Differential Revision: D27880928
fbshipit-source-id: e00a5580a339fdabc4b858235da9cd7e9fc7a376
Summary:
hg.py has a hard coded list of what configs it will copy from the source repo to the remote repo object (hg.py remoteui())
we use one of lfs and one of infinitepush option in edenscm/hgext/clienttelemetry.py where the remote repo object is used
Reviewed By: ahornby
Differential Revision: D27906275
fbshipit-source-id: d551934437126fdd0b920354bf4c51a6e09bafb2
Summary:
Integration test showing that MismatchedHeads errors are sent over the wire to
the client.
Reviewed By: quark-zju
Differential Revision: D27798555
fbshipit-source-id: b14a213e9055486bf07ecbb4b5453385df701b48
Summary:
This error occurs when the client sends us a set of heads that we cannot match.
For example when the client forces a commit in the master group; this was
possible with revlogs but should be a bug with Segmented Changelog. This can
also happen when master moves backwards, clients and server have multiple heads
then the server reseeds.
Clients that get this error should reclone.
Reviewed By: quark-zju
Differential Revision: D27786602
fbshipit-source-id: 9854ccee929ae0a845236ebd83ed68158c93fc7a
Summary:
We want to distinguish between no location and failure to compute location.
It is useful to know on the client side if we failed to compute the location of
some hash or we are not sending it because we don't have a location for it.
We have different options for hash-to-location in particular but the problem is
a general one. Does it make sense for us to skip sending error at the EdenApi
handler layer? I don't think so, I think that's an old hask. We should be
evolving the EdenApi handler layer to handle application errors.
This is still at the exploration phase. I haven't fixed the client side to
handle the errors.
Reviewed By: quark-zju, kulshrax
Differential Revision: D27549926
fbshipit-source-id: 7094160fe997af0e69c8d006b9731ea7dfb3e3f8
Summary:
The goal here is to add an easy way of propagating application errors from
server to the client.
The most convenient form for an error message is a message string so we just
start with that. I think the most important thing to add is some way of
communicating whether the error is retryable or not.
With conversions for anyhow::Error to WireError and from WireError to
ServerError it should be trivial for application code to pass application
errors down to the client.
The format here is driven by the fact that we have streams in the response
object. A batch oriented format for responses has more options. For example
with batches it is common to have a response object that has 3 categories:
1. found: HashMap, 2. not_found: Vec, 3. errors: Vec/HashMap
Reviewed By: kulshrax
Differential Revision: D27549923
fbshipit-source-id: 33b790253adc4761ea9ac7caced4148f4026e620
Summary:
On Mac this introduces 150ms delays in every indexedlog flush. During
an amend of a stack with 32 commits, this fsync accounted for 2/3rds of the time
spent.
Since commitcloud is pretty reliable these days, I think this is no longer
necessary.
Reviewed By: andll
Differential Revision: D27896589
fbshipit-source-id: a13a494c54ffea5a670ed942b366620619af2bd0
Summary: This is needed for rendering multiple progress bars in `cmd.exe`.
Reviewed By: andll
Differential Revision: D27887601
fbshipit-source-id: 9aa987cef327de91408f2e38b4d2e551fe10e39b
Summary:
`distutils_rust` tries to use `mt` to merge the "long path aware" Windows
manifest. It tries to deal with 2 cases:
- The exe already has an embedded manifest. Use `-inputresource` to merge them.
- The exe does not have an embedded manifest. Do not use `-inputresource` since
it will cause an error.
It does so by first passing `-inputresource`, then drop it if the error message
contains "The specified resource type cannot be found in the image file".
However, the error message is translated. Depending on system locale setting,
`mt` might output:
mt : general error c101008c: Failed to read the manifest from the resource of file "hg.exe". The specified resource type cannot be found in the image file.
mt : general error c101008c: Failed to read the manifest from the resource of file "hg.exe". 指定的映像文件不包含资源区域。
So let's check the error code `c101008c` instead.
Reviewed By: andll
Differential Revision: D27885096
fbshipit-source-id: 3d42a3b88ae593434e7104c7899bf6b4f3366180
Summary: `CaseSensitivity::Sensitive` is better than a mere `true` out of nowhere.
Reviewed By: kmancini
Differential Revision: D27867180
fbshipit-source-id: 39d21d3cc3b70c78c6984c3ddbd65b53520770be
Summary:
This is a baseline packer, to let us experiment with packing.
It chooses the dictionary that gives us the smallest size, but does nothing intelligent around key handling.
Note that at this point, packing is not always a win - e.g. the alias blobs are 432 bytes in total, but the resulting pack is 1528 bytes.
Reviewed By: ahornby
Differential Revision: D27795486
fbshipit-source-id: 05620aadbf7b680cbfcb0932778f5849eaab8a48
Summary:
Let's make sure filenodes are warmed at the same time as hg changesets.
We do it implicitly by deriving them in getbundle, however it results in more
attempts of deriving filenodes, because we'd try to derive on every pull.
This is not a huge problem, but that would mean that we could send more
requests to the root filenode to check whether filenodes for a commit were derived or not.
And I remember looking at the logs a few days ago and seeing a few filenodes errors in our
scuba table, which I suspected might be related to the fact that filenodes are
not in derived data cache.
There's a high chance i'm wrong about the root cause of these problems this change should be
reasonable to do anyway.
Reviewed By: krallin
Differential Revision: D27822519
fbshipit-source-id: c878206a09bd19e4a68327dddf27aae10490740f
Summary:
Instead of using get_public method which queries bookmarks let's call
get_public_raw instead, which just does a phases fetch from a local db.
See previous diff for more motivation
Reviewed By: krallin
Differential Revision: D27821547
fbshipit-source-id: a71c8c9ad283259e9be98e63c9c72428e35c6142
Summary: `%s` with a rev number is not accepted in the current codebase. Use `%d` instead.
Reviewed By: ikostia
Differential Revision: D27874102
fbshipit-source-id: b83c40b7182da8639e82bea7bd00a036be4120c4
Summary: `%s` with a rev number is not accepted in the current codebase. Use `%d` instead.
Reviewed By: ikostia
Differential Revision: D27873899
fbshipit-source-id: b34eb0b80f0789c9e06af366bfdaa884c5c69357
Summary: `%s` with `revid` is not accepted in the current codebase.
Reviewed By: ikostia
Differential Revision: D27873898
fbshipit-source-id: e3790855892d3b07e1e5ea6bd92a14738bf6c100
Summary:
We didn't log it to perf counters log, and that makes it hard to aggregate,
show distributions etc
Let's start doing that
Reviewed By: krallin
Differential Revision: D27856968
fbshipit-source-id: 82fbba70154ee011073f3122256bd296bbb938ae
Summary: Its more efficient to bulk load the mappings for a chunk than do the queries one by one
Differential Revision: D27801830
fbshipit-source-id: 9c38ddfb1c1d827fc3028cd09f9ad51e3cbee5dc
Summary: Add an accessor so that we keep a reverse mapping of the WalkState::bcs_to_hg member as a cache of bonsai to hg mappings and also populate it on derivations.
Differential Revision: D27800533
fbshipit-source-id: f9b1c279a78ce3791013c3c83a32251fdc3ad77f
Summary: Add an accessor so that we can use the WalkState::hg_to_bcs member as a cache of hg to bonsai mappings
Reviewed By: farnz
Differential Revision: D27797638
fbshipit-source-id: 44322e93849ea78b255b2e3cb05feb8db6b4c7a7
Summary: This diff makes treeoverlay the default overlay type for Windows users.
Reviewed By: kmancini
Differential Revision: D27247658
fbshipit-source-id: 866eafc794eff1c262eab3061f14eb597bea0a66
Summary: This diff allows EdenFS to create tree overlay based on checkout configuration.
Reviewed By: kmancini
Differential Revision: D27242580
fbshipit-source-id: d0ebe33017c16517c117c1886f62bc9c6fe9f466
Summary:
`debugresetheads` is expected to remove all non-essential heads. That
includes bookmarks.
Reviewed By: kulshrax
Differential Revision: D27861548
fbshipit-source-id: 045976a5a9e27e7eee7ee48448c44552da439983
Summary:
Now that lastCheckoutTime is a single uint64_t, we no longer need a lock to
protect it, simple atomics are sufficient. Since reading an atomic usually
doesn't require any atomic operation, this will save a handful of atomics when
loading an inode where the last checkout time is read.
Reviewed By: chadaustin
Differential Revision: D27860653
fbshipit-source-id: 464e950c949ca243664d213da99d96ff5d0fcbb8
Summary:
The lastCheckoutTime is mostly used to initialize the timestamps of newly
loaded inodes and since these store an EdenTimestamp, we incur a conversion for
every inode loads. Instead of doing this conversion, we can simply make it an
EdenTimestamp directly.
Similarly, the getNow function was always converted to an EdenTimestamp
(sometimes more than once), we can also make it return the EdenTimestamp
directly.
Reviewed By: chadaustin
Differential Revision: D27860652
fbshipit-source-id: 9ea8fe9a312e6c3d8667b93130bb32a46ab92961
Summary:
Some test runner don't properly redirect stdout/stderr of nested processes, or
even direct writes to filedescriptors. On these, debugging a test failure is
almost impossible for EdenFS as we rely on the test output to be interleaved
with the EdenFS logs to understand what the daemon is doing.
To solve this, we can simply create a thread that redirects the output of
EdenFS to sys.std{out,err}.
Reviewed By: kmancini
Differential Revision: D27570966
fbshipit-source-id: 6a8d5229d8d5d141e6ab423f7658744b42af46e3
Summary: The Python `[auth]` matching code does not take cert validity into account when performing certificate matching, whereas the Rust version of the code does. In practice, the existing call sites for the Rust code disable match-time validation, and instead validate the certificate at time-of-use. This diff makes the Rust code's behavior match Python so we can remove the latter entirely.
Reviewed By: DurhamG
Differential Revision: D27837343
fbshipit-source-id: 0bfb5ebc3a36c8fa748cb289e2d8d1495ba8b296
Summary:
The svfs might have a different permission setup (ex. g+s, on ext4) that cannot
be applied to other vfs (ex. on edenfs). Do not inherit it. Instead, calculate
proper mode from the vfs root (ex. `.hg`).
Practically, the `createemode` is `None` in most of our repos. However,
`debugsegmentclone` might create svfs with `g+s` mode due to indexedlog's
`mkdir -p` recursively chmods created parents.
The original logic was added in 6590bef21 (FBS), 80e51429cb9a (HG) in 2008 with
little review comments: https://www.mercurial-scm.org/pipermail/mercurial-devel/2008-July/007269.html
Reviewed By: DurhamG
Differential Revision: D27860581
fbshipit-source-id: 43f93080621aaef168d2cecae46fd6dfb879fa1c
Summary: Enables the `Serialize` and `Deserialize` impls on the `Uuid` type.
Reviewed By: dtolnay
Differential Revision: D27799952
fbshipit-source-id: 4b0e2f8ab4ede20a2113fc1dda42c2ba8b3d0b35
Summary:
Previously we were always caching bookmarks, but D27323369 (f902acfcd1) accidentally remove
that. Let's add it back
Reviewed By: krallin
Differential Revision: D27859523
fbshipit-source-id: 8137c838fc56ecbbc64ba139d4a590dccd011bbc
Summary:
I am debugging why some people get vim to pop up during a merge conflict and some do not.
also fixes a few lint issues
Reviewed By: DurhamG
Differential Revision: D27684419
fbshipit-source-id: f636d71c18353a3816d1e442c05790cf4fd7b90b
