Summary:
Plumb the read-only mount flag into FUSE on macOS. This fixes a build
warning and enables the --read-only mount flag.
Reviewed By: wez
Differential Revision: D20634119
fbshipit-source-id: a5e68cd163e36ceb6d86fd753844718ee9a5727f
Summary:
Instead of explicitly running results on the Thrift thread manager
executor, use the semifuture implementation of the Thrift method which
does the same thing automatically.
Reviewed By: fanzeyi
Differential Revision: D20481540
fbshipit-source-id: 5f23c0193514fce20d80e3292bfa7402f4448e38
Summary: the exception object is unused in each of these catch blocks.
Reviewed By: chadaustin
Differential Revision: D20562251
fbshipit-source-id: 12502429e47f5603b73cfc88b10dda0db5daeb93
Summary:
This is a rough pass that resolves a linker issue on MSVC by
switching to inline static member functions.
Reviewed By: chadaustin
Differential Revision: D20529163
fbshipit-source-id: 578ed440758c685091d3e039e261638e027db17a
Summary:
the type of `mode_t` that we were using for this on Windows
recently changed in the folly portability layer, resulting in a padding/alignment
problem that breaks our assumption at the size of this struct.
I don't believe that we were using bitfields correctly here; in my experience
all of the members must have the same type in order to get packed in the
struct correctly, because the standard doesn't define much in the way
of bitfield behavior beyond the syntax.
This commit uses an unsigned integer type for all of the bitfields
and takes a stab at using a cast when returning the actual mode
member for macOS. This may not be right; I'm just throwing up this
diff first in order to be able to get it on a mac machine.
Reviewed By: simpkins
Differential Revision: D20529006
fbshipit-source-id: e9362bf263fab03d51cc8aa97f94f00806650618
Summary:
s/CURLE_SSL_CONNECT_ERROR/CURLE_RECV_ERROR/.
Note CURLE_SSL_CONNECT_ERROR is not explicitly checked in
code in the eden hierarchy (or anywhere else significant in fbcode).
Reviewed By: meyering
Differential Revision: D20490844
fbshipit-source-id: be4a66f49eb4a9eabaf73785f9a203f0aa6905a4
Summary: This diff bumps the open call from FUSE to High priority (which is higher than any other blob open request atm). This has shown improvement on the user experience of EdenFS when it's importing many other things from other channels (thrift, etc.)
Reviewed By: chadaustin
Differential Revision: D20287389
fbshipit-source-id: 319bc44ef8be5c904d7cf0db7cc2f8be28b4760a
Summary:
Replacing `HgBackingStore` with `HgQueuedBackingStore`.
This does not really improve anything except that all the caller of `getBlob`/`getTree` on this store is generating a `HgImportRequest` and getting a `SemiFuture` from the Promise associated with the request back.
Reviewed By: chadaustin
Differential Revision: D19184822
fbshipit-source-id: a8aef6b0d7392a6c407d311e8e1982754e736e9f
Summary: This diff implements `HgQueuedBackingStore` that uses `HgImportRequestQueue` to provide SCM data importing with priorities. This can allow us to customize how we import things and batch importing.
Reviewed By: chadaustin
Differential Revision: D19184826
fbshipit-source-id: da579b5bbff0b1449e9689e2c0159d4a3a475a83
Summary: This diff adds `HgImportRequestQueue` that is responsible for managing incoming requests by their priorities. This queue is later used in the `HgQueuedBackingStore` to prioritize works to the workers of the backing store.
Reviewed By: chadaustin
Differential Revision: D20197069
fbshipit-source-id: 246bbc086054a8021226e9ba6ab26d3bf0cfb7a3
Summary:
This class is used to represent an import request that will be used later in the queue implementation.
When the EdenFS needs to import a blob, it creates an instance of this request and send it to the worker. Then it waits for the promise associated with the request.
In the future, we should be able to change the owned `Promise` into a non-owned `SemiFuture` to a `Promise` somewhere else for merging repetitive import requests.
Reviewed By: chadaustin
Differential Revision: D19184824
fbshipit-source-id: 823aabbed1156acf6306b7aefc76580a540d310d
Summary: This diff adds `Priority` added in the previous diff to the `BackingStore` interface with the default value set to `Priority::Normal`.
Reviewed By: chadaustin
Differential Revision: D20197071
fbshipit-source-id: a92f1b49bb82e3478042e5e3b79b047d834755ea
Summary:
This diff introduces a `Priority` type for EdenFS. This type is used to pass along the priority of a request.
The priority class itself contains two parts, `kind` and `offset`. `kind` uses the first 4-bytes and the reset 12-bytes are used to store offset. The idea is that we can roughly assign a priority kind to most of the requests and offset is used to dynamically tweak the priority of some particular requests. For example, when we saw a process is generate millions of requests we can use this to express "normal priority but less important than other process's normal priority".
Reviewed By: chadaustin
Differential Revision: D20287652
fbshipit-source-id: 9a849fb6cc6ba5e443fea978d5b4dc3ab8ca906a
Summary:
In dev mode, the glob benchmark failed inside of
folly::Range::operator[] because asserting null termination
technically violates the bounds check.
Reviewed By: simpkins
Differential Revision: D20268416
fbshipit-source-id: ee9b16a6eb9882e850631aa9d83fffe7b6fb67c3
Summary: We should disallow `checkout` calls to queue. In practice, this is hard to encounter because of the Mercurial repository lock, but edge cases can slip through (user running a long `hg up` and control-cing the command and then running `hg up` again). There is not a use case in which `checkout` calls should queue, and keeping the Thrift queue clear of unnecessary (possibly) expensive commands should generally help user experience.
Reviewed By: simpkins
Differential Revision: D19894583
fbshipit-source-id: 2b6f0d6bd1f801872fa6fa66dc7728dc9c83d158
Summary: Missed cleaning up the other definition in D19956272.
Reviewed By: chadaustin
Differential Revision: D20199689
fbshipit-source-id: 9a00cb7b640f71edc034632e354fb61112a74bc1
Summary: Moving these two files to integrate them better with rest of the Inodes code.
Reviewed By: simpkins
Differential Revision: D19956267
fbshipit-source-id: 5bf8165a6dc70a0c03f71f3f73b383991f21eb56
Summary: Windows sdk doesn't define mode_t. On Windows we use mode to store the file type info which should fix in 16 bits - S_FMT is defined as 0xf000.
Reviewed By: simpkins
Differential Revision: D19956270
fbshipit-source-id: 81acd2cd708f1c82674b8b75cd607d4f5b41a156
Summary:
Migration from Future-returning executor-erasing collectX forms to
SemiFuture-returning forms, that are less risky in particular with coroutines.
Earlier diffs added SemiFuture and Unsafe versions. This codemod migrates
collect versions to the Unsafe versions to allow the basic collect versions to
be made safe.
Reviewed By: simpkins
Differential Revision: D20331206
fbshipit-source-id: efc8dff487d45f7d53ee55e8c4696bd3eed0e6da
Summary:
The result of `errnoStr` is often converted to `std::string` so returning `fbstring` adds an extra copy. Make it return `std::string` instead. This will also allow removing dependency between `String.h` and `FBString.h`.
(Note: this ignores all push blocking failures!)
Reviewed By: yfeldblum
Differential Revision: D20195395
fbshipit-source-id: 0dc65f1566911156be3fcb715dd105c58f2a8822
Summary:
Add a counter to report the number of mounts that we failed to remount during
startup. Mount failures do not prevent EdenFS startup from proceeding. It is
useful to have a metric to report if these errors did occur even though the
start-up as a whole still proceeded otherwise.
Reviewed By: chadaustin
Differential Revision: D20319512
fbshipit-source-id: fd503a1ccc91b476cc9dc2bc6323501bbbeaf2c5
Summary: Pass a valid ObjectFetchContext down into certain untracked requests.
Reviewed By: chadaustin
Differential Revision: D20243575
fbshipit-source-id: e7112c3bab1265803a26130c4d72905c25f2e729
Summary: expose the counters for number of pending imports (blobs, trees, prefetches) to allow use in tooling
Reviewed By: chadaustin
Differential Revision: D20269853
fbshipit-source-id: d2b7e2110520290751699c4a891d41ebd5b374cf
Summary:
adds a counter to track the imports queued to enable more statistics exposure.
- Add a counters to track the number of blob, tree, prefetch imports that are in the pending
- have the counters increment (increment in constructor of wrapper struct) when the import is about to be queued
- have counters decrement once the load has completed (decrement in destructor of wrapper struct)
Reviewed By: chadaustin
Differential Revision: D20256410
fbshipit-source-id: 5536b46307b30fc19dc5747414727a86961c78e1
Summary: - added logging only around the import tree call to capture non-queue related wait time
Reviewed By: chadaustin, fanzeyi
Differential Revision: D20207472
fbshipit-source-id: d88bb34ce224a26ff2be100d7789ddeff608006d
Summary:
- added logging only around the import blob call to capture non-queue related wait time
- added to `test_reading_file_gets_file_from_hg` in `integration.stats_test.HgBackingStoreStatsTest` to test import blob logging in addition to the get blob loging
(not yet done for importing trees, will do in next diff)
Reviewed By: chadaustin
Differential Revision: D20201215
fbshipit-source-id: c89281fe7d3d6e89d111ac8cce9014adff44ac40
Summary:
Call `folly::setThreadName()` in the privhelper process when it starts. This
changes the command name reported in `/proc/PID/comm` and in `ps`
The process name is limited to 15 bytes, so this shows up as `edenfs_privhelp`
Reviewed By: fanzeyi
Differential Revision: D20199409
fbshipit-source-id: a5349bfab9230174aaa99c87f0db73fe31659186
Summary:
A bunch of files include folly/executors/GlobalExecutors.h transitively through thrift/lib/cpp2/async/Stream.h, which is going away. Explicitly include the header (and add dependency to target) in preparation for deleting Stream.h
drop-conflicts
Reviewed By: vitaut
Differential Revision: D20141838
fbshipit-source-id: 21c58cf82136287fc2d84ba5badec6b872106015
Summary:
Update the thrift `getDaemonInfo()` call to also return the fb303 status.
This allows the CLI to make a single thrift call instead of 2 when checking if
the EdenFS daemon is healthy.
Reviewed By: genevievehelsel
Differential Revision: D20130406
fbshipit-source-id: 9d25341e1d5f82fb1a921e1d7b1ebd34bcf19dc8
Summary:
The problem is that the datapack files are not flushed to disk when it is prefetched. By having a pair of brackets around the `HgBackingStore`, it will ensure the `HgImporter` is closed by the time when we verify the prefetch with `hg cat` since it will terminate the `debugedenimporthelper` process in its destructor, which flushes the datapack files.
The real cause of the test failure is still unclear but I believe this is the correct way of doing this test.
Reviewed By: xavierd
Differential Revision: D20090249
fbshipit-source-id: 8e3966936a402c92311919433282027846d065e8
Summary: Windows SDK doesn't define dirent. Defining it here for adding Inodes support on Edenfs on Windows.
Reviewed By: simpkins
Differential Revision: D19956272
fbshipit-source-id: 1bdf9a7563c194fe38008741b09668242ffa64ee
Summary:
Logging on Windows doesn't work when the async is set. We haven't debugged it yet. Removing the async mode flag until we fix that.
Also bumping up the log level to 4. This would help to get more info while we are running in beta.
Reviewed By: simpkins
Differential Revision: D19776609
fbshipit-source-id: ccd6a6ed4d81f4a2edd550c6bb7195ac8b8b4d16
Summary:
Most of EdenFS's main logging is done through folly::logging, however a number
of libraries that we use do logging through glog. Previously we set glog's
`--minloglevel` setting to `0`, and we use the default `--v=0` setting.
This enabled glog `VLOG` messages, only for at VLOG level `0` messages.
Now that the Rust backing store code can fetch directly from memcache this now
links in some additional memcache library code that has some `VLOG(0)`
messages that are logged fairly frequently. These aren't useful for us to
have in our logs, so reduce the `minloglevel` to `1` for now, which disables
all `VLOG` messages.
Reviewed By: genevievehelsel
Differential Revision: D20050589
fbshipit-source-id: 167e301d61e46ae3c19975e0c9233eda371495c0
Summary: so that it can be shared more easily with AsyncUDPSocket
Reviewed By: yangchi
Differential Revision: D19851480
fbshipit-source-id: ec8cdb852519724db6f89cf70c4a4169de5028b6
Summary: It seems to be stable and not causing issues. Let's make it default everywhere.
Reviewed By: wez
Differential Revision: D19896738
fbshipit-source-id: cf6abe8f536e570017742b3a0674213a932a6a4d
Summary:
Eden SCM expects that DRY_RUN reports the same conflicts as a normal
checkout, but EdenFS would skip traversing deleted trees in dry run
mode. Fix that and add a test.
Reviewed By: genevievehelsel
Differential Revision: D19782543
fbshipit-source-id: 7a269e67a41b7ad6ce6c54fde37e8f74fcc1ef51
Summary: Notifications is using folly Subprocess which doesn't work on Windows.
Reviewed By: genevievehelsel
Differential Revision: D19863375
fbshipit-source-id: 63b047253c0f8a48b1b0ccc767f5820e77a28d80
Summary: records if a start was successful or not
Reviewed By: simpkins
Differential Revision: D19817810
fbshipit-source-id: b67253099781bb534b7e2fb26a09ba41c1f0bd69
Summary: On Windows we will verify both Windows and POSIX path separators, because we have both types of paths.
Reviewed By: simpkins
Differential Revision: D19562772
fbshipit-source-id: a6d8280e3bae4f6cd32b1f379bb59e22dd584211
Summary: There is a bug in WinStore::getTreeEntry(). It fetches the shared_ptr to the Tree and returns the raw pointer to the entry in the Tree. Using the entry without holding the Tree shared_ptr is unsafe. This bug was introduced while refactoring the code in D19377522.
Reviewed By: simpkins
Differential Revision: D19762486
fbshipit-source-id: 6f0a849f0dc9731bada9a56bdc2a6c18740b0b18
Summary: logs information about daemon shutdowns, including duration, if it was a graceful restart or not, and if it was successful or not.
Reviewed By: fanzeyi
Differential Revision: D19817811
fbshipit-source-id: a851de8872f98952d046fb148a27fbf9f05b2212
Summary:
This commit manually synchronizes the internal move of
fbcode/scm/mononoke under fbcode/eden/mononoke which couldn't be
performed by ShipIt automatically.
Reviewed By: StanislavGlebik
Differential Revision: D19722832
fbshipit-source-id: 52fbc8bc42a8940b39872dfb8b00ce9c0f6b0800
Summary:
This commit causes a desktop notification to be shown if we generate
EIO or ETIMEDOUT responses via fuse; the prompt is intended to make it obvious
to the user that they need to connect to the VPN.
The commit by itself doesn't show a notification, it allows configuring a
command that can be run to do something to show a notification.
The test plan includes one such configuration for our corp environment.
* It doesn't trigger for thrift-originated downloads (eg: prefetch), only for
VFS operations through FUSE.
* Ideally we'd know exactly when we have a network related error in the store
code and use that to trigger the notification. However, we have a rather
convoluted set of importers and fallbacks today, one of which is interpreting
a generic response returned from a pipe, so it is not especially clear
exactly where we should locate the logic
Reviewed By: chadaustin
Differential Revision: D17513364
fbshipit-source-id: 45134f3672679cb5580cb0c1bc12a0d6e38525ca
