Summary: Starting from 3.11.1, OSXFUSE switched into using macOS's major version number for different system versions. So we need to consider that when calculating path to the kernel extensions on macOS.
Reviewed By: xavierd
Differential Revision: D25675984
fbshipit-source-id: ea8c76ce7204ba5da3ca98ceca2cfbeb9c84fa8f
Summary:
Make sure we give more explanation to users to they can self-fix any errors
related to certificates that might pop up.
Reviewed By: xavierd
Differential Revision: D25758517
fbshipit-source-id: 3b9929be3d1c0c44a5e13cc9c1e7b2a4f785abf4
Summary:
The introduction of `eden trace` broke the Buck build on Windows due to its use
of streaming thrift which unfortunately doesn't compile on Windows. Since `eden
trace` is not supported on Windows for now, let's only depend on the streaming
thrift on Linux and macOS.
With this, we can now compile edenfsctl on Windows with Buck. This will later
enable integration tests to be run on Windows.
Reviewed By: genevievehelsel
Differential Revision: D25758445
fbshipit-source-id: d4be2cafd9472840f65dcfab63a5fcfb8eceffb7
Summary:
Like it says in the title. Judging by an earlier similar change (D21092866 (15f98fe58c)),
this kind of flakiness in walker tests occurs when a node's children are
reachable via other paths.
Reviewed By: HarveyHunt
Differential Revision: D25756891
fbshipit-source-id: 05bc0697381e068d466ea6dfe85529dbd9ef1a50
Summary:
Like it says in the title. Note that I did *not* retry stuff like resolving
hosts or connecting, so this should only really temporary blips in
connectivity. We probably shouldn't go much beyond that at a low level like
this.
Reviewed By: HarveyHunt
Differential Revision: D25615915
fbshipit-source-id: 78c33eff2e9ce380a260708e9fbeb929eede383c
Summary:
This is the goal of this stack: retry errors that occur when Curl detects that
the transfer speed is too low. This should let us eventually set a much higher
timeout on overall request completion, thus ensuring that we don't uploads
that make progress, all the while aborting connections early and retrying them
if they are legitimately stuck.
Reviewed By: farnz
Differential Revision: D25615790
fbshipit-source-id: fe294aee090758b1a3aef138788ac2926c741b79
Summary:
Right now, the error handling in LFS doesn't handle e.g. transfer timeouts. I'd
like us to support that, notably so that we can have curl require a minimum
transfer speed and retry if we fail.
To do so, I need to be able to capture the errors and figure out if they're
retryable. Right now, everything is either a `FetchError` that includes a HTTP
status and URL, or just an `Error` that aborts.
This diff introduces a `TransferError` that explains why a transfer failed and
can be used for retry decisions. We later add the request details if we decide
to not retry and turn it into a `FetchError`.
Reviewed By: xavierd
Differential Revision: D25615789
fbshipit-source-id: e4a2f4f16a34ca2f86bd61491bb26e7f328dec63
Summary:
Like it says in the title. This adds support for setting a min-transfer-speed
in Curl. My goal with this is to fix two problems we have:
- a) Uploads that timeout on slow connections. Right now we set a transfer
timeout on requests, but given files to upload can be arbitrarily large, that
can fail. This happened earlier this week to a user (T81365552).
- b) Transfer timeouts in LFS. Right now, we have a very high timeout on
requests and we can't lower it due to this problem with uploads. Besides,
the reason for lowering the timeout would be to retry thing, but right now
we don't support this anyway.
Reviewed By: xavierd
Differential Revision: D25615788
fbshipit-source-id: 57d75ee8f522cf8524f9d12103e34b0765b6846a
Summary:
I'd like to make it a little easier to add more options without having to
thread them all the way through to the HTTP transfer callsite.
Reviewed By: xavierd
Differential Revision: D25615787
fbshipit-source-id: 4c6274dc2e6b5ba878e0027aae9a08b04f974463
Summary: Extended git-import test to include both `full-repo` and `missing-for-commit` import modes.
Reviewed By: ahornby
Differential Revision: D25675361
fbshipit-source-id: b93e2db963c2060540308bf0477cd891d40e5810
Summary:
Managing tailer processes that run multiple times and run once is different. We
want separate code paths when we run contiously than when running only once.
Reviewed By: quark-zju
Differential Revision: D25684782
fbshipit-source-id: 354b32c1dd73f867d6a7b1bd4518d9dd98e6b9a3
Summary:
The intention was to sort entries by Dag Id entry. This was instead sorted
lexicographically.
Reviewed By: quark-zju
Differential Revision: D25684784
fbshipit-source-id: 0a3db6398aec7d8df080bbb2366e41660483608c
Summary:
Comments for why we don't need a lock when updating the SqlIdMap with multiple
writers. Structure can definitely be improved but I'll live with this for a
short time.
No fundamental change in logic. I added extra checks to the insert function and
changed from an optimistic insert race logic to a pessimistic version. I
explain in the comments that it's to have an easier time reasoning about what
happens and that theoretically doesn't matter.
Reviewed By: quark-zju
Differential Revision: D25606290
fbshipit-source-id: ea21915fc797fe759b3fe481e8ad9e8cb594fb6a
Summary: Will reduce the number of jobs needed for small repos
Reviewed By: StanislavGlebik
Differential Revision: D25492059
fbshipit-source-id: de11c06615857ad43f3337e58973849d2026a114
Summary: preparation for multi repo, get the repo name into ErrorKind::NotTraversable
Reviewed By: StanislavGlebik
Differential Revision: D25541444
fbshipit-source-id: 8fd99d5d3f144d8a3a72c7c33205ae58bd5f1ae2
Summary:
In preparation for having the walker able to scrub multiple repos at once, define parameter structs. This also simplifies the code in tail.rs.
The param objects are:
* RepoSubcommandParams - per repo params that can be setup in setup_common and are consumed in the subcommand. They don't get passed through to the walk
* RepoWalkParams - per repo params that can be setup in setup_common and will get passed all the way into the walk.rs methods
* JobWalkParams - per job params that at can be setup in setup_common and will get passed all the way into the walk.rs methods
* TypeWalkParams - per repo params that need to be setup in the subcommands, and are passed all the way into walk.rs
Reviewed By: StanislavGlebik
Differential Revision: D25524256
fbshipit-source-id: bfc8e087e386b6ed45121908b48b6535f65debd3
Summary: parsing of progress options an sampling options was same in each subcommand, move them to functions in setup.rs
Reviewed By: StanislavGlebik
Differential Revision: D25524255
fbshipit-source-id: a2f48814f24aa9b3a158cb7d4abbfc2c0c338305
Summary: Simplify open_blobrepo_given_datasources parameters to pass less arguments, make it so can pass the sql_factory by reference.
Reviewed By: krallin
Differential Revision: D25524254
fbshipit-source-id: c324127f42c53a52f388d303e310014f4fa0d7bb
Summary: Allows the walker blobstore code to be used by more than one blobrepo. This is a step to reduce the number of jobs needed to scrub small repos.
Reviewed By: StanislavGlebik
Differential Revision: D25422937
fbshipit-source-id: e2d11239f172f50680bb6e10dd60026c9e6c3c3d
Summary:
By doing the hg to hg steps via bonsai I will later introduce a check if the bonsai is in the current chunk of commits to be processed as part of allowing walker checkpoint and restart.
On its own this is a minor change to the number of nodes the walk will cover as seen in the updated tests.
Reviewed By: krallin
Differential Revision: D25394085
fbshipit-source-id: 3e50cf76c7032635ce9e6a7375228979b2e9c930
Summary: This is in preparation for all walker hg to hg steps (e.g HgChangeset to Parent HgChangeset) going via Bonsai, which without this would continually check if the filenodes are derived
Reviewed By: krallin
Differential Revision: D25394086
fbshipit-source-id: bb75e7ddf5b09f9d13a0f436627f4c3c95e24430
Summary:
`PartialOrd` was suggested by sfilipco. Note `Option<std::cmp::Ordering>` is
similar to `Side` in terms of expressiveness. `PartialOrd` can be written
using shorter symbols (`<=`, etc) so it's easier to understand.
The `compatible` family APIs were replaced by `partial_cmp` APIs.
There are some minor differences:
- Bitwise or used by union set is no longer supported. `Hints::union` was
added as a replacement.
- `Option<T>` implements full order. `Some(T) > None`. This is different
from `compatible_dag` and `compatible_id_map` APIs. Additional `> None`
checks were added for correctness.
Reviewed By: sfilipco
Differential Revision: D25652784
fbshipit-source-id: 51d88948fa556300678050088c06e9dda09cbf98
Summary:
```
warning: variable does not need to be mutable
--> eden/scm/lib/configparser/src/config.rs:448:21
|
448 | let mut values_copy = values.clone();
| ----^^^^^^^^^^^
| |
| help: remove this `mut`
|
= note: `#[warn(unused_mut)]` on by default
```
Reviewed By: sfilipco
Differential Revision: D25625453
fbshipit-source-id: 8475056a87095f9ba633282666e6d3fee864074b
