Summary:
The bind-mounts configuration has been ignored by EdenFS since D17236366.
This removes all CLI code for dealing with this config section.
Reviewed By: wez
Differential Revision: D20876460
fbshipit-source-id: 6b3f3552de25ee28fc0418a6aaec14446520203c
Summary:
In rare situations users end up manually deleting or removing their `.eden`
state directory without ever killing their running `edenfs` process. This can
leave this old process running indefinitely despite the fact that it's state
directory is no longer present (or has perhaps even been replaced with new
data).
This updates edenfs to periodically check if its lock file is still valid, and
quit if it isn't. This will help prevent old `edenfs` processes from running
indefinitely after their state directory is no longer valid.
Reviewed By: wez
Differential Revision: D20613841
fbshipit-source-id: d9a3a1e7e9b05806e086e794ebbc36e1cc71831a
Summary: It seems to be stable and not causing issues. Let's make it default everywhere.
Reviewed By: wez
Differential Revision: D19896738
fbshipit-source-id: cf6abe8f536e570017742b3a0674213a932a6a4d
Summary:
This commit causes a desktop notification to be shown if we generate
EIO or ETIMEDOUT responses via fuse; the prompt is intended to make it obvious
to the user that they need to connect to the VPN.
The commit by itself doesn't show a notification, it allows configuring a
command that can be run to do something to show a notification.
The test plan includes one such configuration for our corp environment.
* It doesn't trigger for thrift-originated downloads (eg: prefetch), only for
VFS operations through FUSE.
* Ideally we'd know exactly when we have a network related error in the store
code and use that to trigger the notification. However, we have a rather
convoluted set of importers and fallbacks today, one of which is interpreting
a generic response returned from a pipe, so it is not especially clear
exactly where we should locate the logic
Reviewed By: chadaustin
Differential Revision: D17513364
fbshipit-source-id: 45134f3672679cb5580cb0c1bc12a0d6e38525ca
Summary:
cpptoml has traversal functionality for table reads, but not for
writes. Add a helper function for reading a config value and updating
the TOML table if it's unset.
Reviewed By: fanzeyi
Differential Revision: D19671264
fbshipit-source-id: e2b78d338af35d51fddaa258b7f45f8966d00a26
Summary: Easier to zero initialize structs with braces, which defines that even padding is zeroed.
Reviewed By: wez
Differential Revision: D19655675
fbshipit-source-id: 2fd12383324029646707e93008cf9ad34e9f1dce
Summary: This code is no longer used now that `eden redirect` exists.
Reviewed By: wez
Differential Revision: D19565010
fbshipit-source-id: 4703bc39a024c4c491a83ef5a6e85711a8f7e4d8
Summary:
Instead of clearing every single cached object when the total size
exceeds the ephemeral storage limit, keep a limit per object type and
only clear those that exceed their quota.
Reviewed By: simpkins
Differential Revision: D19358312
fbshipit-source-id: 6918d6f4cc2931aed79a9025d0e0f357ede515e0
Summary:
Add a config option to restrict the number of tree prefetches that can run in
parallel. Without this applications that recursively walk a checkout tree can
end up spawning a huge number of asynchronous prefetch operations. These
prefetch operations are quite expensive today, as we currently have to fetch
the full file contents in order to retrieve the file sizes. A recursive
directory walk can end up building up a huge backlog of in-progress tree
prefetches. This will slow down the directory walk, and it can take tens of
minutes to finish processing the prefetch backlog even after the directory
walk is aborted.
Reviewed By: chadaustin
Differential Revision: D19136685
fbshipit-source-id: cdc7a570d016fd7ca81a639cef83380b197acdfc
Summary:
Add an option `experimental:use-edenapi` to `EdenConfig`.
See the next diff for usage.
Reviewed By: chadaustin
Differential Revision: D18605549
fbshipit-source-id: 2786c21bb38a76229078662cc5c1ddf906d1be4a
Summary:
I think we need to make `EdenConfig` available for testing as more things are using `EdenConfig` to configure behaviors.
Right now it does not do much things other than just construct an `EdenConfig` instance without manually providing these parameters.
Reviewed By: chadaustin
Differential Revision: D18798484
fbshipit-source-id: b4a85d966a9b4f425c547bef9eb6e7570d7e2170
Summary:
This splits `EDEN_BUG()` into three separate version. All three crash in
debug mode builds, but in release builds they behave differently:
- `EDEN_BUG()` throws an exception
- `EDEN_BUG_FUTURE(Type)` returns a `folly::Future<Type>` that has been
fulfilled with an exception.
- `EDEN_BUG_EXCEPTION()` returns a `folly::exception_wrapper`.
The main advantage of this is that this allows the compiler to detect that
`EDEN_BUG()` can never return. Previously `EDEN_BUG()` was used for all 3 of
these different cases, and its behavior depended on whether `toException()`
was ever called. As a result we could not easily get the compiler to identify
code paths where we know at compile time that it will never return.
Reviewed By: chadaustin
Differential Revision: D18652103
fbshipit-source-id: 070107c7520f51b05696905fa243de5f8df15958
Summary: Adding an experimental option to EdenFS so we can disable blob caching if we need.
Reviewed By: chadaustin
Differential Revision: D18441665
fbshipit-source-id: 56751c0199d6658bfbf7ec3865f746a7279324ab
Summary: This reads `enforceParents` from a config instead of always assuming true for `getScmStatusV2()`. This will allow a easy kill switch in case throwing errors from this thrift call causes issues with something that calls hg status
Reviewed By: simpkins
Differential Revision: D18258164
fbshipit-source-id: 1ae421a941c01a678d25d5453c771262b03558d0
Summary:
If a checkout's SNAPSHOT file disappeared, the mount error given would
be that a Hash had an invalid size. Instead, throw a file not found
error on mount.
Reviewed By: genevievehelsel
Differential Revision: D18381074
fbshipit-source-id: 35282e0990189d4084a2a64330a5733561cf88c2
Summary:
Background: https://fb.workplace.com/groups/sourcecontrolteam/permalink/2440123159442349/?comment_id=2443624539092211
This may or may not be the root cause but this is definitely gonna be causing issues.
Between D17468473 and D17866320, turning on this option will make EdenFS to use the old Rust code that does not strip copyrev data.
Reviewed By: wez
Differential Revision: D18250237
fbshipit-source-id: b3816bd08b8dc3b2a90931829c368a58731c7843
Summary:
The Windows build spews a great many warnings. Address some of them by
enabling the unused-exception-parameter warning on Clang/Linux too.
Reviewed By: yfeldblum
Differential Revision: D18178930
fbshipit-source-id: efecb605b84d4f06c8c8411a23d17904bbdff746
Summary:
Introduce a framework that allows recording structured log events
which are encoded as JSON and piped to a configured command line
program.
Reviewed By: pkaush
Differential Revision: D18025183
fbshipit-source-id: ab6b4d510a905a30252f2cff85d107a0d32d149e
Summary: While reading the config code, I made some minor, behavior-changing improvements.
Reviewed By: genevievehelsel
Differential Revision: D17918029
fbshipit-source-id: e40bce099a9555559c028c701f6fb75a6a4fc6be
Summary:
Add a configuration knob that allows environments to opt out of Eden's
custom permission checks on the unix domain socket, instead using the
standard unix domain socket permission rules for the host OS.
Reviewed By: wez
Differential Revision: D17863456
fbshipit-source-id: c60c52891e49cc8027832bd42029cfd52752547b
Summary:
Instead of having accessors for every config setting in EdenConfig,
just expose the ConfigSettings directly.
Reviewed By: fanzeyi
Differential Revision: D17847805
fbshipit-source-id: 8c6c1010c010113cf859677449797ea916f2a2a5
Summary:
I plan to simplify access to EdenConfig by making ConfigSettings
public rather than writing accessors for each setting. To avoid
exposing implementation details, switch to private inheritance.
Reviewed By: wez
Differential Revision: D17847752
fbshipit-source-id: 02ef7afe96f09cc29a54b21bfafbef9234e6f74d
Summary:
I found the logic for whether the config should be reloaded
unnecessarily complicated, so reduce it to deciding whether to reload
or not. This removes the need to acquire a write lock in autoreload's
common case.
Reviewed By: wez
Differential Revision: D17847699
fbshipit-source-id: 50fee1aac15cc8f896333c93459fea6510646600
Summary: This allows us to test the datapack code easier without rebuilding Eden.
Reviewed By: wez
Differential Revision: D17468473
fbshipit-source-id: a6807b4d6e747ae8557ae51fdf798de2a54fd4f1
Summary:
The code in `EdenConfigChecks.cmake` had two separate `find_package(cpptoml)`
calls, one using its installed CMake config file, and one using a custom
`Findcpptoml.cmake` module.
This removes the custom `Findcpptoml.cmake` code, and updates everything to
only used cpptoml's installed CMake configuration.
Reviewed By: chadaustin
Differential Revision: D17401220
fbshipit-source-id: 3789703cdfc029049db3b1bd9f5751fa2a60a8d4
Summary:
Update the CMakeLists.txt to also build the Python-based `edenfsctl` command
line tool.
This requires switching most of the thrift rules to generate both C++ and
Python sources.
Note that one missing feature at this point is that this does not package
external dependencies into the binary. Currently `edenfsctl` depends on both
`six` and `toml` as external dependencies. For now these must be available in
your `PYTHONPATH` in order to run the generated `edenfsctl` binary.
Reviewed By: chadaustin
Differential Revision: D17127615
fbshipit-source-id: fc138ab39e75c6a5bbd39e3f527d4e9f7f420e46
Summary:
This fixes a few issues with the library dependencies:
- The `eden_utils` library depends on `eden_service_thrift`, not
`eden_service`. By incorrectly depending on `eden_service` this introduced
a circular dependency which would cause a build failure, depending on which
order CMake chose to try and emit the link line.
- The `eden_config` library depends on code from `eden_model` (for `Hash` and
`ParentCommits`)
- The `eden_inodes` library depends on `eden_model_git` for the `GitIgnore`
logic. I also alphabetized the dependency list.
Reviewed By: wez
Differential Revision: D17124930
fbshipit-source-id: 70cbe81081fc1dc807cca13a93edc25ba270b01f
Summary:
Rename the `ThriftCppLibrary.cmake` file to `FBThriftCppLibrary.cmake`, and
also rename `add_thrift_cpp2_library()` to `add_fbthrift_cpp_library()`.
Explicitly calling this `fbthrift` helps clearly distinguish that this is
intended for use with fbthrift (https://github.com/facebook/fbthrift/), as
opposed to Apache thrift.
Reviewed By: wez
Differential Revision: D16738440
fbshipit-source-id: 9b255e06b71c98ad74a34989f564a211958dcdd5
Summary:
This diff adds a fuse request timeout configuration setting
and threads it through to both the FuseChannel for our internal processing
and to the privhelper so that we can set an appropriate (slightly larger)
value for the kernel level daemon_timeout setting.
Reviewed By: chadaustin
Differential Revision: D16957552
fbshipit-source-id: a0fecc691d72914b5aebaed8a006dc0d6b0d7d12
Summary: This code calls close on fd while exiting. If the file doesn't exist the open will return -1 and close (-1) will be called.
Reviewed By: strager
Differential Revision: D15951485
fbshipit-source-id: ea3a52517847d75e9a822e51f360be7cb2c411da
Summary:
Add a periodic task for performing LocalStore management tasks. For now only
the RocksDBLocalStore class implements this management task.
When this periodic task runs the RocksDBLocalStore object computes how much
space each of the column families are using and publishes this as fb303
counters. If the total size of the ephemeral column families exceeds a
configurable limit it then triggers a background garbage collection task.
I also added a new `edenfsctl stats local_store` command that reports the new
counters added by this diff.
Reviewed By: chadaustin, strager
Differential Revision: D15798505
fbshipit-source-id: 25ca4ba80f5a9c4a1a09dc08633c7b3af363d7ff
Summary:
Update the copyright & license headers in thrift files to reflect the
relicensing to GPLv2+
Reviewed By: wez
Differential Revision: D15487082
fbshipit-source-id: 33f68617037f36c07075fb962a16a4d8f55bd6a6
Summary:
Update the copyright & license headers in CMake files to reflect the
relicensing to GPLv2
Reviewed By: wez
Differential Revision: D15487079
fbshipit-source-id: 715e559464c19a0070d6e55a095b3fc7d61ad2f8
Summary:
Update the copyright & license headers in C++ files to reflect the
relicensing to GPLv2
Reviewed By: wez
Differential Revision: D15487078
fbshipit-source-id: 19f24c933a64ecad0d3a692d0f8d2a38b4194b1d
Summary:
Add a periodic task to reload the configuration file from disk. By default
this runs once every 5 minutes, but this interval can be controlled from the
config file.
At the moment reloading the config file does not do much other than update the
interval for how frequently the config file is reloaded. However, I plan to
add additional periodic tasks shortly that are controlled by this config
setting.
This will also make it possible for other parts of the code to
access the config settings in the `ServerState` and use them as-is without
checking to see if they reloaded. Currently all of the code that accesses
config values performs a check to see if the config needs to be reloaded. If
we want to switch to Mercurial-style configs in the future that check will be
substantially more expensive.
This diff also includes a new thrift call to force the config file to be
reloaded immediately. This can be used to restart automatic config reloading
if it is ever disabled in the config file.
Reviewed By: wez
Differential Revision: D15756357
fbshipit-source-id: 1999f4730903633ce838842932a6ae6a65eda4e6
Summary:
The `FieldConverter<bool>` code previously relied on `folly::to<string>()` for
all arithmetic types, including `bool`. Unfortunately `folly::to<string>()`
returns `"1"` and `"0"` for booleans, which isn't terribly human-friendly.
This updates the code to explicitly return `"true"` and `"false"` as the
output for converting boolean values to strings.
Reviewed By: strager
Differential Revision: D15550014
fbshipit-source-id: 69a3385e529e2940a8be20dbcbeda92e5836d969
Summary:
Add a `FieldConvertor` implementation allowing a config field to be parsed
into a `std::chrono::nanoseconds` value.
Reviewed By: strager
Differential Revision: D15428932
fbshipit-source-id: d3ace45a2986717c798897534e7b3015eda35fa9
Summary:
The `ConfigSetting` code previously used `folly::to<string>()` to convert
field values back to strings for debugging. However, this only works for
values that actually are supported by `folly::to()`
This changes the code to use the `FieldConverter` to perform the conversion
back to string. The `FieldConverter` class now contains the logic for doing
conversion both from strings and back to strings.
Reviewed By: strager
Differential Revision: D15428933
fbshipit-source-id: a37a978d151bf0ba79ca2a2b9107401c847b6591
Summary:
Update the `FieldConverter` code to support parsing any arithmetic type
(integers and floating point) using `folly::to<T>()`. Also change the `bool`
conversion implementation to use the same code.
This does change the `bool` implementation to accept any string accepted by
`folly::to<bool>(string)`. The old code only allowed `true` and `false`. The
new code allows other strings: `yes`, `no`, `y`, `n`, `0`, `1`, `on`, `off`
This also makes the implementation more efficient by eliminating some
unnecessary string copies and avoiding throwing exceptions internally on
error.
Reviewed By: wez
Differential Revision: D15428700
fbshipit-source-id: feb73a1f53775fbf3d3acb8585fa689f79c0db81
Summary:
Add a thrift call to get the current config settings.
My primary use case for this method at the moment is to make it possible to
build integration tests that check the config behavior. However in the future
this will probably also be useful for building CLI commands to report the
current config values to allow debugging if there are ever issues. This API
can also be used to force EdenFS to immediately reload the config from disk.
Reviewed By: strager
Differential Revision: D15572124
fbshipit-source-id: da3bc982f9c419b3314a8b0560c9bd327760d429
Summary:
Change the `ReloadableConfig::getEdenConfig()` method to accept an enum to
control whether it should reload the configuration, rather than the simpler
bool that it used previously.
The primary motivation for this is to allow us to force a reload to occur in
unit tests and integration tests, even if this operation would otherwise be
within the normal reload throttling time limit.
Reviewed By: strager
Differential Revision: D15572122
fbshipit-source-id: 9c638a415a451f26e7f5923f544724594ed4e0f1
Summary:
Move the ConfigSource enum definition to a thrift file. This will let us
return ConfigSource values over thrift APIs in the future. This also allows
us to use thrift's `TEnumTraits` functionality to determine the maximum enum
value, rather than having to maintain a separate `kConfigSourceLastIndex`
variable.
As part of this change I also renamed the enum values to be CamelCase to match
our current C++ style recommendations and to avoid possibly conflicting with
macros defined in other headers (`DEFAULT` seemed particularly susceptible to
collision).
Reviewed By: strager
Differential Revision: D15572120
fbshipit-source-id: 8fbd03da221a9f75ef670dee1eb250eb198a5bd0
Summary:
This solves the client certificate mystery. We creates a new `EdenConfig` object
whenever there is a change detected and these two fields are lost during the
copying. Causing the variable substitution not working as expected.
Reviewed By: wez
Differential Revision: D15221180
fbshipit-source-id: 3fba36c8f261cb3dbe25e2e363d65661240be8cf
Summary: Change `ConfigSource` from a plain `enum` to an `enum class`.
Reviewed By: chadaustin
Differential Revision: D15162811
fbshipit-source-id: 680ec1e785de1a54b623d99477914afa5d0931ca
Summary:
Move the `ConfigSetting` template and related classes from `EdenConfig.h` to a
new `ConfigSetting.h` header file.
Reviewed By: chadaustin
Differential Revision: D15162815
fbshipit-source-id: 34fba0b8b0e8e50714f622bae642d20d77050673
