Summary:
This commit manually synchronizes the internal move of
fbcode/scm/mononoke under fbcode/eden/mononoke which couldn't be
performed by ShipIt automatically.
Reviewed By: StanislavGlebik
Differential Revision: D19722832
fbshipit-source-id: 52fbc8bc42a8940b39872dfb8b00ce9c0f6b0800
Summary:
Attach the content to the exception object directly so if the exception is
handled, no verbose message will be printed.
Reviewed By: DurhamG
Differential Revision: D19817404
fbshipit-source-id: 9b79952559d07eb62bd7eb2398c6dd022ffe7b28
Summary:
I think I might want to call this from more than one place in
a later diff, so extract it into a helper function.
Reviewed By: chadaustin
Differential Revision: D19770166
fbshipit-source-id: e044003736c6ba21984a9129da1df50ce92b2f35
Summary:
This commit causes a desktop notification to be shown if we generate
EIO or ETIMEDOUT responses via fuse; the prompt is intended to make it obvious
to the user that they need to connect to the VPN.
The commit by itself doesn't show a notification, it allows configuring a
command that can be run to do something to show a notification.
The test plan includes one such configuration for our corp environment.
* It doesn't trigger for thrift-originated downloads (eg: prefetch), only for
VFS operations through FUSE.
* Ideally we'd know exactly when we have a network related error in the store
code and use that to trigger the notification. However, we have a rather
convoluted set of importers and fallbacks today, one of which is interpreting
a generic response returned from a pipe, so it is not especially clear
exactly where we should locate the logic
Reviewed By: chadaustin
Differential Revision: D17513364
fbshipit-source-id: 45134f3672679cb5580cb0c1bc12a0d6e38525ca
Summary:
Include the full error source chain on any internal errors. This will improve
debugability when internal errors are encountered, as the outermost context
might not be enough to describe the origin of the error.
Detect when backtraces are disabled, and only include them on the error if a backtrace was captured.
Reviewed By: StanislavGlebik
Differential Revision: D19813785
fbshipit-source-id: c28ffc0c44050a82cb7050afa57e976f37962432
Summary: create a bounded traversal crate with the 0.1 version of bounded_traversal_stream functionality, prior to updating to futures 0.3
Reviewed By: farnz
Differential Revision: D19804873
fbshipit-source-id: e98e00111fee5b1a9fcfc20bb054eeae1263fb26
Summary: replace use of spawn_future with tokio 0.2 join handle
Reviewed By: krallin
Differential Revision: D19770171
fbshipit-source-id: e0b7bf3da58896b223149b339a72bfec997215ee
Summary:
Update the walker step methods to use new futures, and combine them with async fn
Later in stack planning to:
* remove use of spawn_future and replace it with the tokio 0.2 join handles
* port bounded_traversal_stream to new futures so all these 0.3 futures don't immediated get compat'd back to 0.1
Reviewed By: farnz
Differential Revision: D19767902
fbshipit-source-id: 10fd6236a064efbb7d0815fadbdd32036bcafead
Summary:
D19767626 added an original_timestamp column to the
blobstore_sync_queue. Update the sqlite schema to keep it in sync.
Reviewed By: krallin
Differential Revision: D19787488
fbshipit-source-id: ad576e2ec99349953e2ab69e3defb73d1ff556c0
Summary:
Modify the multiplexed blobstore implementation so that the
multiplex_id is written to the healer queue after a put. Further, update the
blobstore healer to only look at entries with the same multiplex ID as it's
configured to run with.
Reviewed By: ahornby
Differential Revision: D19770057
fbshipit-source-id: 41db19f0b0f84c048d49ab9e6258cccc89cf4195
Summary:
This test deliberately races itself. Unfortunately, tokio's scheduler is sufficiently quick that if we spawn all the futures as they're created, they sometimes don't race each other.
Fix this by spawning in the join line instead.
Reviewed By: ahornby
Differential Revision: D19812651
fbshipit-source-id: 86685bbb71c451e9c2a96100c83ddff28d0718dd
Summary:
Before we start blocking generation of derived data let's start with logging if
derived data is not specified.
Reviewed By: farnz
Differential Revision: D19791523
fbshipit-source-id: 15bed8463f8a021de76a2878f06ec95d9fef877f
Summary:
See D19787960 for more details why we need to do it.
This diff just adds a struct in BlobRepo
Reviewed By: HarveyHunt
Differential Revision: D19788395
fbshipit-source-id: d609638432db3061f17aaa6272315f0c2efe9328
Summary:
Looks like most of our tests got slower recently, and this particular test
started failing as a result since it's sensitive to timing. Unlike when this
test was written, we can now get a little more info from Mononoke LFS by
looking at Scuba logs, to know if we went to upstream or not. So, let's do
that.
Reviewed By: HarveyHunt
Differential Revision: D19790000
fbshipit-source-id: 5617b088595c911018166d2c13eb43dc6adca60b
Summary: Make the EdenAPI server report that it is exiting when asked to shut down. This ensures that Proxygen will stop sending traffic to servers that are about to be shut down by Tupperware. This diff is basically the same as krallin's diff D17626009 for the LFS server.
Reviewed By: quark-zju
Differential Revision: D19782432
fbshipit-source-id: 41b9e6761145402e7dcf18c53a2b33799588594c
Summary: This diff sets up the Mononoke API (from the `mononoke_api` crate) in the EdenAPI server, and makes it available to route handlers by adding it to a new `EdenApiContext` struct that is maintained as part of the server's global state. The server's route handlers should use the Mononoke API for accessing source control data, and any new source control business logic should be incorporated into that crate rather than being part of the EdenAPI server itself.
Reviewed By: xavierd
Differential Revision: D19778441
fbshipit-source-id: bc2efb82e0276d75c49980c52fa0d3017a4ce2f1
Summary:
See also https://github.com/serde-rs/bytes/.
This will be used in the `dag` crate.
Reviewed By: DurhamG
Differential Revision: D19770858
fbshipit-source-id: 2a870a564e0ceecdc7a4667853b2b2a5ea4ce6e3
Summary:
This crate provides the core features of the commonly known `Bytes` crate:
zero-copy slicing and cloning, while also supports mmap-backed buffers.
The main motivation is to replace `Mmap` in `indexedlog`. That has multiple
benefits:
- Handles 0-sized mmap more cleanly.
- Handles clones more cleanly.
- Gain the flexibility to zero-copy data without lifetime / reference.
- Gain the flexibility to switch to non-mmap data.
The `bytes::Bytes` crate does not yet support mmap buffers as of its latest
release (0.5.4).
Implementation wise, `minibytes::Bytes` uses `Option<Arc<dyn Trait>>` for the
"trait object". This makes implementing the mmap storage just one line.
`bytes 0.5.4` re-invents the "trait object" manually using unsafe code. It requires
about 50 lines to implement the mmap storage (in D19756122).
Reviewed By: xavierd
Differential Revision: D19770856
fbshipit-source-id: 8cfa7052a18ac2e0cd6348b77d5e2a4acc61195c
Summary:
Our main data table has no information about the actual failure. Let's
upload a stack trace to blob storage and link to it from the main table.
Reviewed By: quark-zju
Differential Revision: D19780653
fbshipit-source-id: acaab29a47d32dd6ab81fed0c3ae50c8eb44b760
Summary:
If the main process forks, the connection to the cache client is likely
unusable. Let's drop that connection and we can recreate it later if needed.
Reviewed By: quark-zju
Differential Revision: D19796387
fbshipit-source-id: 59b0d3db9655d2233b55effcdf18cbd74a3f5edd
Summary: All the crates are present in third-party/rust, let's use it in the OSS build instead of fetching a tarball of all the crates.
Reviewed By: fanzeyi
Differential Revision: D19770783
fbshipit-source-id: f0d74bb0807be207d599d4868f907d38099c7f5b
Summary:
We're seeing an error where worker processes fail to fetch data from
the server. Our theory is that the forking messes up the ssh connection, so
let's not reuse ssh connections across forks.
Reviewed By: xavierd
Differential Revision: D19794281
fbshipit-source-id: 3954cd96617fd18ecf0d0aa9b9a6ce774c494067
Summary:
In D19581174 I made `--debug` trigger ipdb session for `.t` tests.
This diff adds similar feature for `.py` tests.
Reviewed By: DurhamG
Differential Revision: D19784795
fbshipit-source-id: 9118c74ea465320f15aa6ad5e1e04d8bc1a49966
Summary:
Currently existing validation won't catch a bug where commits `a <- b` get
replayed as `b <- a` as long as they don't touch the same files. Let's add
such check.
Reviewed By: StanislavGlebik
Differential Revision: D19723150
fbshipit-source-id: ddc15063b9ae4fc38416ab9b96681da302fec8d4
Summary: after reviewing my stack D18647089 , chadaustin recommended some documentation surrounding this area since it is a bit more complex now, and since it was easy to deadlock and there are a lot of moving parts, I added documentation about how the diff path works, including the new code path, and other non-trivial parts.
Reviewed By: chadaustin
Differential Revision: D19688435
fbshipit-source-id: f6512340421e7c25d4fb4d27177c6187eb57ccbf
Summary:
In order to uniquely identify a blobstore multiplexer configuration,
add an ID.
Reviewed By: krallin
Differential Revision: D19770058
fbshipit-source-id: 8e09d5531d1d27b337cf62a6126f88ce15de341b
