Summary:
This and following diff will refactor CheckoutPlan creation.
Right now we create CheckoutPlan from manifest diff and then manipulate it with methods like `with_sparse_profile` to adjust plan for different cases.
Those 'adjustment' do not work great with the structure of CheckoutPlan, for example `with_sparse_profile` has to create temporary HashSet just to index files in CheckoutPlan
We are going to add more adjustments in the future (for example, checkout --clean), and will run into same issues with current structure of CheckoutPlan
To avoid those issues, we are going to refactor this code, so that instead of Diff -> CheckoutPlan -> adjustments, we are going to have Diff -> ActionMap -> adjustments -> CheckoutPlan
The structure of CheckoutPlan is still good for it's direct purpose (execution), but all the 'changes' to the plan will be done in ActionMap instead.
Reviewed By: DurhamG
Differential Revision: D27980390
fbshipit-source-id: 403f371fd2fe7760984925a38429e1bfb88d8e3f
Summary: For now this does not work with --clean flag(fallback to regular checkout in that case)
Reviewed By: quark-zju
Differential Revision: D27953967
fbshipit-source-id: 71c097cf1e395ff2cba2f4ee528145d3b2c83c23
Summary: This crate does not calculate status, but allows to convert python status into rust status, that can later be used by python code
Reviewed By: quark-zju
Differential Revision: D27953962
fbshipit-source-id: ab91d9d035140e43d8b17988b24bd030af77c96d
Summary: When checking out on dirty copy without --clean this function can be used to check if checkout operation conflicts with currently modified files
Reviewed By: quark-zju
Differential Revision: D27953965
fbshipit-source-id: 4096506e4cbf8b102e0afa1a929c066dfa474825
Summary:
This crate introduces consumer API for status in rust
Currently the implementation will just take status from Python and convert it into this struct
But in the future we can get pure Rust implementation to get status
Reviewed By: quark-zju
Differential Revision: D27953963
fbshipit-source-id: 29c876400c82056eaf81fffa4adc814473853c1e
Summary: This method can be used to 'normalize' path for case insentive use cases
Reviewed By: quark-zju
Differential Revision: D27953964
fbshipit-source-id: 421832af22af9a3b56eec0d045b9f983592ed192
Summary:
Fix missing stats for the bookmarks endpoint because we have the wrong name in
code.
Reviewed By: quark-zju
Differential Revision: D28008091
fbshipit-source-id: 128fe00e00a06ebe9b65fb11512cd03a042d55fe
Summary:
This doesn't link the errors into Scuba, which makes it not very useful for
debugging, since we're not routinely looking at stderr on our tasks, and makes
it impossible to e.g. look anywhere and find a count of failed requests.
Instead, update this to use `capture_first_err`, which will report both the
first error and the total error count to Scuba.
Reviewed By: sfilipco
Differential Revision: D27998037
fbshipit-source-id: b941d44a2ac21bbf640b9bc977de749207f12d9a
Summary:
In EdenAPI, most endpoints don't raise errors when they fail, and instead just take items out of the response stream (though there are some exceptions: D24315399 (0ccae1cef9).
Right now, this just gets logged to stderr, which isn't great. This diff introduces a CaptureFirstError wrapper we can use to capture those errors and expose them in post-response callbacks (i.e. Scuba).
Reviewed By: sfilipco
Differential Revision: D27998036
fbshipit-source-id: 960d0e09a82ca79adfafe22e2eeef2e0294d27dc
Summary:
`lld` is faster than `ld.gold`.
To compare, `make local3`, then add a blank line in `hgmain`, then run `make
local3` again.
Using `lld`:
building 'hgmain' binary
Compiling hgmain v0.1.0 (/home/quark/fbsource-lazy/fbcode/eden/scm/exec/hgmain)
Finished release [optimized + debuginfo] target(s) in 3.73s
building 'mkscratch' binary
Finished release [optimized] target(s) in 0.37s
building 'scm_daemon' binary
Finished release [optimized] target(s) in 0.42s
Using `ld.gold`:
building 'hgmain' binary
Compiling hgmain v0.1.0 (/home/quark/fbsource-lazy/fbcode/eden/scm/exec/hgmain)
Finished release [optimized + debuginfo] target(s) in 10.15s
building 'mkscratch' binary
Finished release [optimized] target(s) in 0.39s
building 'scm_daemon' binary
Finished release [optimized] target(s) in 0.46s
Reviewed By: ikostia
Differential Revision: D28006551
fbshipit-source-id: 55b19be93d8152634d79ed92c9cd53237f91b820
Summary:
This raw XDB connections API is not used in Mononoke although there are projects depending on it.
The API creates connection objects based on shed/sql crate.
This diff moves `SqlConnections` into `shed/sql:sql_common`, closer to the `Connection` definition, and moves `create_raw_xdb_connections` API into the `shed/sql:sql_facebook::raw`.
Reviewed By: krallin
Differential Revision: D28003638
fbshipit-source-id: ea4a29b6e239a89c97237877e2dfde4c7c7ff89b
Summary:
It turns out during the initial clone, we're not using selectivepull for some
tiers that do not have the non-repo selectivepull config.
We've been using selectivepull for devservers and corp (and it's effective
during clone) for a long time. Let's just enable it by default so even if
dynamicconfig does not set it properly, we can still use selectivepull clone
to avoid pulling 10k+ remote bookmarks (which triggers auto bookmark cleanups
as logged in hgfeatures).
There are too many incompatible tests so I'm not migrating them for now.
Reviewed By: DurhamG
Differential Revision: D28006488
fbshipit-source-id: f0dc000156abde530fd8881bd26b4642a36167be
Summary:
As I split the pushrebase crate in D27968029 (93b8cf116b) it makes sense to stop re-exporting hook definitions from it.
This change will also make dependencies more accurate.
Reviewed By: krallin
Differential Revision: D28028045
fbshipit-source-id: 622ef5d7db737e19153109f4d2dcefe54fba2bb4
Summary:
Finally! This is basically the end goal of this stack (though we still need to
do the same thing with the "ForwardErr" combinator used by EdenAPI next).
With this, we can now capture errors that occur while sending the response
stream, instead of just errors that occur while producing the response (which
is usually a little piece of the work we have to do).
Reviewed By: mitrandir77
Differential Revision: D27967991
fbshipit-source-id: a5483c58f0550a19e711e712cf860d9328a0eb9e
Summary:
`ContentMeta` sounds a lot like a struct containing content metadata, so let's
rename this to something less ambiguous (`ContentMetaProvider`).
The reaosn I'm doing this now is because I'd like to introduce a similar
trait for errors that occur on our response streams and there I'll need
both a struct and a trait.
Reviewed By: mitrandir77
Differential Revision: D27998039
fbshipit-source-id: f0372c62d13167ef4bd07cb9e9e9fb75ea105b9a
Summary:
Like it says in the title. The goal here is to make it not matter where the
error came from. In this diff, we capture the same errors as before, but we
do it via PostResponseInfo instead of via ad-hoc things in our `State`.
Reviewed By: mitrandir77
Differential Revision: D27967994
fbshipit-source-id: dbbb1a8f5ea1a439089c41b4a08cd6088476ae33
Summary: Like it says in the title.
Reviewed By: mitrandir77
Differential Revision: D27967992
fbshipit-source-id: 0deb4d90538a6889bee6b41de4c5d1533b29519b
Summary:
Very small refactor. I want this stuff to all be in the same module instead
of spread across `response` and `error`.
Reviewed By: mitrandir77
Differential Revision: D27967993
fbshipit-source-id: aca22f952d756d298e5e342f0c4f8ebd31f108bf
Summary:
This is a bit of an abstract change, but the goal of this change is to make
post-response handlers oblivious to the distinction between sending a response
(or failing to send one) and returning a response that actually contains a
(fallible) stream.
The underlying / ultimate goal here is to move our error reporting out of
ad-hoc router wrappers where we call `set_error_message` on some context
entity, and to instead move it into post-response callbacks.
The upshot of that is that if we fail to send a response even though we sent a
200 from the handler, we'll be able to log it! Indeed, remember that when
sending a streaming response, we have to send a 200 to start streaming but we
might hit an error producing later parts of the response!
Reviewed By: mitrandir77
Differential Revision: D27966422
fbshipit-source-id: ab49639bfcc4af23ddc2e84181278f105ebb28b9
Summary:
This stuff runs after we sent the response, so PostResponse is a more
appropriate name than PostRequest.
Reviewed By: ikostia
Differential Revision: D27966420
fbshipit-source-id: 1f7b7a55490f731eb512793024bcfafb0ea4ef79
Summary:
Those two have historically used different (but largely copy pasted) code to
produce their responses. Let's unify them by
While in there, let's also modernize the formatting a little bit by letting
anyhow do the formatting for us (we used to use `failure` when this code was
written, and it didn't do it).
There's a bit of ugliness here in the sense that out formatting is injecting
the error into the state so it can be logger later. This is equivalent to what
we had, but it's kinda clonwy. That said, I'm working on refactoring our error
handling in this stack, so the plan is to clean this up (i.e. it won't stay
very long).
Finally, note that this diff removes the `ResponseCreationFailure` error kind
in LFS. This is replaced by a `.context()` in `gotham_ext`. That said, we never
really use this stuff: creations are fallible in Hyper but you only run into
an error if you e.g. forget to add a status code, so we don't expect them to
actually occur outside of development.
Reviewed By: mitrandir77
Differential Revision: D27966421
fbshipit-source-id: 097f3b69f25fe39f8fbef925a272e88199896b39
Summary:
Like it says in the title, I'd like to use names that reflect that this isn't
just *any* content stream: it's specifically for responses.
Reviewed By: ahornby
Differential Revision: D27964045
fbshipit-source-id: 50530cf85ba7840a2baa14151351d0b288d9ae70
Summary:
We have a set of things that are meant to be used together that are spread
across 3-4 different modules. Let's move them together. This also allows us to
make some things (e.g. the `ContentMeta` trait) private as they're no longer
needed.
Note: this diff only move stuff around & renames things a bit. I'll also split
some of those modules in the next diff.
Reviewed By: HarveyHunt
Differential Revision: D27964047
fbshipit-source-id: 02528d84adfd70ec346c32163cb185d89266a53e
Summary:
We have a module called "content" that contains two completely unrelated
things: some enums for content encodings (+ associated parsing), and our output
streams.
I'd like to add more of said output streams, so let's clean this up.
Reviewed By: HarveyHunt
Differential Revision: D27964046
fbshipit-source-id: b42e56aa3fadaf9b93a44216977da19d950a16b9
Summary:
We used to have to do this because of overly strict trait bounds in Hyper, but
we no longer do, so let's get rid of it. Note that we have one Tokio task per
request, and polling the response stream is basically the only thing that task
does, so this should make little difference as far as task scheduling is
concerned besides avoiding unnecessary context switches.
Reviewed By: ahornby
Differential Revision: D27963458
fbshipit-source-id: 24e762eb173156dab909fefe11dcf2d58272048a
Summary: We don't use this anymore. Might as well remove it.
Reviewed By: ahornby
Differential Revision: D27963411
fbshipit-source-id: a6ac337936e8b2bd788dd79a835eef11b19dde70
Summary: It has been fixed and we now set auth config with higher priority anyway.
Reviewed By: johansglock
Differential Revision: D28026081
fbshipit-source-id: 7086b48139bb05ffadd782898a1758ae06236aca
Summary:
This change makes it so that our binaries do not instantiate a real configo
client in integration test setup.
Reviewed By: ahornby
Differential Revision: D28026790
fbshipit-source-id: 0fb9ce66a1324e845f4b8a80d4479263ec6e4ee1
Summary:
First, some background on the existing WrappedPath type: In Mononoke the MPath type is such that None==Root and Some(MPath)==NonRoot. This means that where a path may be present one needs to use double-Option with Option<Option<MPath>>, so that Root is Some(None).
To reduce the need for double Option, and subsequently to allow for newtype features like memoization, the walker has WrappedPath, so we can use Option<WrappedPath> instead.
This change introduces a similar type WrappedPathHash for MPathHash, which means that the sample_fingerprint for WrappedPath can be now be non-optional as even root paths/manifests can now have a sample_fingerprint.
Reviewed By: mitrandir77
Differential Revision: D27995143
fbshipit-source-id: b674abd4ec94749f4f5797c697ae7381e1a08d02
Summary:
This adds the first part of new logging from the walker that can be used to gather details on what keys might make sense to pack together.
Unlike the corpus command that dumps file content by path (which was useful for analysis on compression approaches), this adds logging to the scrub command that includes the path hash rather than the full path. This should keep memory usage down during the run, hopefully mean we log from existing scrub jobs and and mean the logs are more compact.
Reviewed By: mitrandir77
Differential Revision: D27974448
fbshipit-source-id: 47b55112b47e9b022f16fbb473cf233a7d46bcf3
Summary:
Knowing the prepushrebase changeset id is required for retroactive_review.
retroactive_review checks landed commits, but verify_integrity hook runs on a commit before landing. This way the landed commit has no straightforward connection with the original one and retroactive_review can't acknowledge if verify_integrity have seen it.
Reviewed By: krallin
Differential Revision: D27911317
fbshipit-source-id: f7bb0cfbd54fa6ad2ed27fb9d4d67b9f087879f1
Summary:
Split pushrebase crate into pushrebase hook definition and pushrebase implementation.
Before this change it was impossible to store an attribute in BlobRepo that would depend on PushrebaseHook as it would create a circular dependency `pushrebase -> blobrepo -> pushrebase`.
Reviewed By: krallin
Differential Revision: D27968029
fbshipit-source-id: 030ef1b02216546cd3658de1f417bc52b8dd9843
Summary:
It's rare (only seem to be used by chistedit) but if revrange got an int, the
expected behavior is to treat it as a revision number.
Reviewed By: kulshrax
Differential Revision: D27983989
fbshipit-source-id: f9f8d9cb39af4ec1de7ed8ca69f7f1879b4a4614
