Summary: so that it can be shared more easily with AsyncUDPSocket
Reviewed By: yangchi
Differential Revision: D19851480
fbshipit-source-id: ec8cdb852519724db6f89cf70c4a4169de5028b6
Summary: It seems to be stable and not causing issues. Let's make it default everywhere.
Reviewed By: wez
Differential Revision: D19896738
fbshipit-source-id: cf6abe8f536e570017742b3a0674213a932a6a4d
Summary:
Eden SCM expects that DRY_RUN reports the same conflicts as a normal
checkout, but EdenFS would skip traversing deleted trees in dry run
mode. Fix that and add a test.
Reviewed By: genevievehelsel
Differential Revision: D19782543
fbshipit-source-id: 7a269e67a41b7ad6ce6c54fde37e8f74fcc1ef51
Summary: Notifications is using folly Subprocess which doesn't work on Windows.
Reviewed By: genevievehelsel
Differential Revision: D19863375
fbshipit-source-id: 63b047253c0f8a48b1b0ccc767f5820e77a28d80
Summary: records if a start was successful or not
Reviewed By: simpkins
Differential Revision: D19817810
fbshipit-source-id: b67253099781bb534b7e2fb26a09ba41c1f0bd69
Summary: On Windows we will verify both Windows and POSIX path separators, because we have both types of paths.
Reviewed By: simpkins
Differential Revision: D19562772
fbshipit-source-id: a6d8280e3bae4f6cd32b1f379bb59e22dd584211
Summary: There is a bug in WinStore::getTreeEntry(). It fetches the shared_ptr to the Tree and returns the raw pointer to the entry in the Tree. Using the entry without holding the Tree shared_ptr is unsafe. This bug was introduced while refactoring the code in D19377522.
Reviewed By: simpkins
Differential Revision: D19762486
fbshipit-source-id: 6f0a849f0dc9731bada9a56bdc2a6c18740b0b18
Summary: logs information about daemon shutdowns, including duration, if it was a graceful restart or not, and if it was successful or not.
Reviewed By: fanzeyi
Differential Revision: D19817811
fbshipit-source-id: a851de8872f98952d046fb148a27fbf9f05b2212
Summary:
This commit manually synchronizes the internal move of
fbcode/scm/mononoke under fbcode/eden/mononoke which couldn't be
performed by ShipIt automatically.
Reviewed By: StanislavGlebik
Differential Revision: D19722832
fbshipit-source-id: 52fbc8bc42a8940b39872dfb8b00ce9c0f6b0800
Summary:
This commit causes a desktop notification to be shown if we generate
EIO or ETIMEDOUT responses via fuse; the prompt is intended to make it obvious
to the user that they need to connect to the VPN.
The commit by itself doesn't show a notification, it allows configuring a
command that can be run to do something to show a notification.
The test plan includes one such configuration for our corp environment.
* It doesn't trigger for thrift-originated downloads (eg: prefetch), only for
VFS operations through FUSE.
* Ideally we'd know exactly when we have a network related error in the store
code and use that to trigger the notification. However, we have a rather
convoluted set of importers and fallbacks today, one of which is interpreting
a generic response returned from a pipe, so it is not especially clear
exactly where we should locate the logic
Reviewed By: chadaustin
Differential Revision: D17513364
fbshipit-source-id: 45134f3672679cb5580cb0c1bc12a0d6e38525ca
Summary: after reviewing my stack D18647089 , chadaustin recommended some documentation surrounding this area since it is a bit more complex now, and since it was easy to deadlock and there are a lot of moving parts, I added documentation about how the diff path works, including the new code path, and other non-trivial parts.
Reviewed By: chadaustin
Differential Revision: D19688435
fbshipit-source-id: f6512340421e7c25d4fb4d27177c6187eb57ccbf
Summary: In addition to duration and success, log object fetch counts and checkout type to Scuba.
Reviewed By: fanzeyi
Differential Revision: D19334276
fbshipit-source-id: dabf52427f2ebda2b58df93194df39d52f4fcb4f
Summary: Log the number of object lookups and cache hit rates for a checkout operation.
Reviewed By: simpkins
Differential Revision: D19191201
fbshipit-source-id: 5e9ad501e704810f072dabcda3fce86d027c452e
Summary:
During checkout and stats, count every object fetch and which level of
cache it was served from.
Reviewed By: simpkins
Differential Revision: D19186333
fbshipit-source-id: fc0a74db297b9c723682e245996a7befd762f933
Summary:
Add a fetch context interface to ObjectStore that allows tracing cache
hits, backing store fetches, and fetch durations in the context of a
diff or checkout operation.
Reviewed By: simpkins
Differential Revision: D19135625
fbshipit-source-id: d0d8f134b1c89f7ba4971a404a46a69a1704ba5c
Summary:
cpptoml has traversal functionality for table reads, but not for
writes. Add a helper function for reading a config value and updating
the TOML table if it's unset.
Reviewed By: fanzeyi
Differential Revision: D19671264
fbshipit-source-id: e2b78d338af35d51fddaa258b7f45f8966d00a26
Summary: This diff updates the states transitions in the EdenMount on Windows. It starts as State::UNINITIALIZED and transitions to State::RUNNING when the start is called. It will transition to State::SHUT_DOWN on stop or destroy. Destroy will put it in State::DESTROYING, from which it should not return.
Reviewed By: chadaustin
Differential Revision: D19559271
fbshipit-source-id: d76983cab610cb9b2c896807cf1fe49c357f8095
Summary:
1fb027d759
changed the kernel behavior to reject reads smaller than 8KB,
even for requests that would never need to be that large.
That causes eden to fail to start up on eg: Fedora 31 with a 5.4 kernel.
This commit adds some padding to satisfy this new check.
Reviewed By: chadaustin
Differential Revision: D19736893
fbshipit-source-id: 926456d72124b186976ee9a8a21242e93c26f790
Summary:
There were some unsafe uses of folly::collect, especially in
the diff code. Switch to collectSafe.
Reviewed By: simpkins
Differential Revision: D19529784
fbshipit-source-id: e0ac920a2d2b44767a453f2c41a9166e72246ab6
Summary:
folly::collect's Future is completed immediately when any of its input
future is completed with an exception. This makes forking off subtasks
and then joining with folly::collect dangerous when closures don't
hold pointers or references to parameters, including `this`. Introduce
a collectSafe function, which has the same signature as folly::collect
but unconditionally waits until all futures are completed.
Reviewed By: simpkins
Differential Revision: D19529772
fbshipit-source-id: b3b493fcbb0d9058d2d6e5c8a064dfdae3a43e57
Summary:
Update the EdenFS monitor to restart itself when it receives SIGHUP. This
allows upgrading the monitor daemon to a new version of the software without
disrupting the current running EdenFS daemon.
Reviewed By: wez
Differential Revision: D19588698
fbshipit-source-id: f0303788cee0cc9bf4ce3b7342f10a595c3874b1
Summary:
Update the SpawnedEdenInstance class to wait for EdenFS to become healthy
before signalling the Future returned by `start()`.
This code works by just polling the `getStatus()` call every few hundred
milliseconds, and waiting until it returns `ALIVE`.
Currently the code will poll forever, until EdenFS either becomes healthy or
exits. In practice we will likely configure startup timeouts in systemd,
which means that they probably are not necessary in this code.
Reviewed By: wez
Differential Revision: D19588699
fbshipit-source-id: fef57ca23fcc14da908c2e4f7f715db33b7bf841
Summary:
Add a simple new wrapper daemon to manage the edenfs daemon. This is intended
to provide a few different features:
- Perform log rotation for EdenFS's output and the output of any of its
spawned children processes.
- Help schedule restarts of EdenFS when the system looks idle.
- Provide a single process for the system to manage across graceful EdenFS
restarts, to make management slightly simpler.
This initial commit does not perform graceful restarts yet, but has the basic
daemon management and log rotation present.
Reviewed By: wez
Differential Revision: D19588700
fbshipit-source-id: bba41c9f7efeb4417753c1d48dd72cf6d191f0c3
Summary: Easier to zero initialize structs with braces, which defines that even padding is zeroed.
Reviewed By: wez
Differential Revision: D19655675
fbshipit-source-id: 2fd12383324029646707e93008cf9ad34e9f1dce
Summary:
When importing an object from the backing store failed, we would not
fail lookup requests (e.g. lstat) so that the file could still be
removed with `rm`. Unfortunately that data would get cached, even if
the error was transient. This diff returns a correct inode number for
failed requests and limits the amount of time these fake stat()
results are cached.
Reviewed By: wez
Differential Revision: D19655425
fbshipit-source-id: 1a44ace9217f1ab3f69a2c5a6edc89d0f9c4a941
Summary:
Multiple times I've gotten mixed up by the different casing of the
LogEvent fields versus the names in the structured log. Change the
field names to match the log names.
Reviewed By: genevievehelsel
Differential Revision: D19585307
fbshipit-source-id: 3ccbb9ec5e18155367bc85e9db00bc8d556812c4
Summary:
I have a suspicion that most edenfs starts are unclean. To test that
hypothesis, add a "mount" structured event type that records whether
the mount was cleanly shut down, how long mounting took, and whether
it's a graceful restart.
Reviewed By: genevievehelsel
Differential Revision: D19585161
fbshipit-source-id: 16eedfeb6181742e64b45bca1ed3cce3edb663cb
Summary: the request will be passed with the context, so simply check the context at all three "looping" points to see if the request was cancelled. I split this into its own diff due to its unique test plan.
Reviewed By: chadaustin
Differential Revision: D18647091
fbshipit-source-id: ccb7ac765f8f3b0a60e936779cc27b658b9fd8b9
Summary:
In any case in which our Inode is not materialized and we would like to continue to diff, we can take the hashes of both the Inode and the ScmEntry and start using the source control tree Differ.
Also, this allows us to remove the RemoveDiffEntry since that only looked at the scmEntry, so that work can also be deferred to the source control tree differ easily.
This avoids loading inodes when we don't need to. In the case of a clean checkout with two commits 30000 commits away, we reduced Inodes loaded by 98.6%.
Reviewed By: chadaustin
Differential Revision: D19292528
fbshipit-source-id: 7b5ec161f7bd65962ae30c49fdad6dbc73954c2a
Summary:
In any case in which our Inode is not materialized and we would like to continue to diff, we can take the hashes of both the Inode and the ScmEntry and start using the source control tree Differ.
This reduces the number of inodes loaded by avoiding to load them if we don't need to. In the case of a clean checkout with two commits 30000 commits away, we reduced Inodes loaded by 98.6%.
Reviewed By: chadaustin
Differential Revision: D19292527
fbshipit-source-id: 2dbf6c4130214f581b6cc09e663f7567abab6ef3
Summary:
In any case in which an inode is not materialized and we are computing a diff, we can take the hashes of both the Inode and the ScmEntry and start using the source control tree Differ.
This avoids loading inodes when not necessary. In the case of a clean checkout with two commits 30000 commits away, we reduced Inodes loaded by 98.6%.
Reviewed By: chadaustin
Differential Revision: D18647087
fbshipit-source-id: 74632de777f04e2468d10977cc757b8993d439e9
Summary: Pass a GitIgnoreStack* and isIgnored flag through the diff operation. It is used later in the stack when we go down this code path in the TreeInode diffing code. When context->loadFileContentsFromPath() is null, the gitignore loading code will be ignored, used when we do not want to honor a gitignore file. For example, during `getScmStatusBetweenRevisions()`.
Reviewed By: chadaustin
Differential Revision: D18647089
fbshipit-source-id: 20d2abd2ef61669465e134165da5a0ac5e987cca
Summary: Produce a -Wdeprecated-declarations warning in code that uses the old SYNCHRONIZED macros.
Reviewed By: yfeldblum
Differential Revision: D19567088
fbshipit-source-id: b0637c06cbd125d09e1b39eec1e82a71ddca2324
Summary:
In the case that the overlay was closed before the initial
createOverlayFile, prevent an invalid OverlayFile from being accessed,
causing the test to fail.
Reviewed By: genevievehelsel
Differential Revision: D19591636
fbshipit-source-id: 9d961f044c3b68b5c0b2dcd108ff85db6326276d
Summary: I reproduced the KeySpace.cpp ICE on my Windows 10 VM, so bump the MSVC version check. I have not confirmed it does not ICE on MSVC 2019.
Reviewed By: pkaush
Differential Revision: D19562325
fbshipit-source-id: 03cf84e95c709bb2ebe7c3e0055a4f290eb17f8a
Summary: This code is no longer used now that `eden redirect` exists.
Reviewed By: wez
Differential Revision: D19565010
fbshipit-source-id: 4703bc39a024c4c491a83ef5a6e85711a8f7e4d8
Summary:
It's not often obvious to people where edenfs.log is located on the
filesystem, so redirect people to `eden debug log` instead.
Reviewed By: simpkins
Differential Revision: D19560709
fbshipit-source-id: 6b031c80bab2c21dfddba4d7ec37d4e70218ea08
Summary:
Now that loading too many blobs only GCs the blob cache, treat cached
trees as ephemeral again.
Reviewed By: simpkins
Differential Revision: D19358375
fbshipit-source-id: 191427f46276f0f5d024b5bc923a68f7a42044da
Summary:
Instead of clearing every single cached object when the total size
exceeds the ephemeral storage limit, keep a limit per object type and
only clear those that exceed their quota.
Reviewed By: simpkins
Differential Revision: D19358312
fbshipit-source-id: 6918d6f4cc2931aed79a9025d0e0f357ede515e0
Summary:
EdenFS no longer uses the blobsize keyspace. To avoid wasting space,
clear and campact it on startup, along with any keyspaces we
deprecated in the future.
Reviewed By: fanzeyi
Differential Revision: D19354881
fbshipit-source-id: 5285757a0e44ab1080c5f940283e06b17bec811d
Summary:
Indicate getManifestEntry is deprecated so we remember to delete it
down the line.
Reviewed By: genevievehelsel
Differential Revision: D19459793
fbshipit-source-id: 86dd173cfb12c36cf6ba19eaf4be96dfe624c05f
