Commit Graph

42 Commits

Author SHA1 Message Date
Katie Mancini
fa180ad585 check multiple locations for x509 certs
Summary:
This stack updates eden to be able to check all of the locations that able
users certificate may reside.

There can be multiple places where a cert may reside (we cant always
definitively choose one place to look based on the platform). Thus we
need to be able to configure multiple locations for certs in our eden
config.

This makes the switch over in eden from using one place for the client
cert to use and using the first available client cert from a list.

NOTE: most of this diff is fixing unit tests take a look at `EdenConfig.h` and `EdenConfig.cpp` first

Reviewed By: wez

Differential Revision: D23359939

fbshipit-source-id: 44beecce3ef098a734dbd7c5eb3fa5f0aad6b50b
2020-09-23 19:58:52 -07:00
Xavier Deguillard
2ff478ea62 utils: move win/utils/Stub.h to utils/NotImplemented.h
Summary:
Since the Stub.h now only contains NOT_IMPLEMENTED, let's move it to its own
header outside of the win directory.

Reviewed By: genevievehelsel

Differential Revision: D23696244

fbshipit-source-id: 2dfc3204707e043ee6c89595668c484e0fa8c0d0
2020-09-16 12:31:46 -07:00
Katie Mancini
903532758e teach eden to resolve THRIFT_TLS_CL_CERT_PATH in configs
Summary:
This stack updates eden to be able to check all of the locations that able
users certificate may reside.

THRIFT_TLS_CL_CERT_PATH is usally set with the location for the users x509
certs. So it seems best to check this location. In order to be able to check
this location, we need to be able to resolve the enviroment variable in our
parsing.

Reviewed By: wez, genevievehelsel

Differential Revision: D23359815

fbshipit-source-id: 2008cc52ab64d23dbcfda41292a60a4bf77a80df
2020-09-15 10:28:38 -07:00
Katie Mancini
9c0be6c7c8 teach eden to parse arrays in configs
Summary:
This stack updates eden to be able to check all of the locations that able
users certificate may reside.

There can be multiple places where a cert may reside (we cant always
definitively choose one place to look based on the platform). Thus we
need to be able to configure multiple locations for certs in our eden
config.

Thus we need to be able to parse a list of options for a key in our config
parsing.

**Disclaimer this is really icky**

Our `FieldConverter` interface takes a string to parse. So this means
that after parsing the config file for each value we have to re-serialize it
into a string to pass it in here. Previously we only supported string and
bool values so this re-serialization was not too terrible. Now that we want
to support arrays this re-serialization is extra gross. To minimize the grossness,
I am reusing cpptoml for serializing / deserializing around the `FieldConverter`
interface.

Long term it would be better if FieldConverter took a cpptoml::base or
something more generic instead of a string so we dont have to do this.
But that will be a big refactor, and I don't currently have bandwidth for it :(

Reviewed By: wez

Differential Revision: D23359928

fbshipit-source-id: 7c89de485706dd13a05adf19df28425d2c1756a8
2020-09-14 17:40:13 -07:00
Chad Austin
d58f0d9451 prevent spurious configuration reloads when a config file doesn't exist
Summary:
getConfigStat had a bug where it, instead of clearing the bits of
*configStat, cleared the bits of the pointer itself. This caused the
stat struct for missing files to be uninitialized memory, causing
configs to reload. Write a test and fix the bug.

Reviewed By: xavierd

Differential Revision: D23645087

fbshipit-source-id: ad42f7ec1b313f668604e3a7f6c8200f6b94b23d
2020-09-13 01:37:14 -07:00
Chad Austin
be7a6b5cb3 introduce an assertZeroBits function to double-check the compiler
Summary:
While hacking on some code, I ran into a situation where some
zero-initialized stat structs weren't actually being zeroed. This was
either a compiler bug or a situation where the build system was not
correctly rebuilding everything after my changes, and I did not have
enough disassembly available to investigate.

Either way, since this code assumes zero bits in some nonobvious ways,
explicitly assert they are.

Reviewed By: xavierd

Differential Revision: D23644819

fbshipit-source-id: eb6bff9ff997379113db1e1bf9d6a0a538f10f0b
2020-09-13 01:37:14 -07:00
Chad Austin
e392eca46e log the path that caused configs to be reloaded
Summary:
We noticed spurious config file reloads, so add some logging to help
track that down.

Reviewed By: xavierd

Differential Revision: D23644447

fbshipit-source-id: 9953a17de402660c7f6491fb9abd8d702fa290e8
2020-09-11 16:53:49 -07:00
Victor Zverovich
e3f4a56f6b Migrate to field_ref Thrift API
Summary:
We are unifying C++ APIs for accessing optional and unqualified fields:
https://fb.workplace.com/groups/1730279463893632/permalink/2541675446087359/.

This diff migrates code from accessing data members generated from unqualified
Thrift fields directly to the `field_ref` API, i.e. replacing

```
thrift_obj.field
```

with

```
*thrift_obj.field_ref()
```

The `_ref` suffixes will be removed in the future once data members are private
and names can be reclaimed.

The output of this codemod has been reviewed in D20039637.

The new API is documented in
https://our.intern.facebook.com/intern/wiki/Thrift/FieldAccess/.

drop-conflicts

Reviewed By: yfeldblum

Differential Revision: D22631599

fbshipit-source-id: 9bfcaeb636f34a32fd871c7cd6a2db4a7ace30bf
2020-07-21 11:23:35 -07:00
Chad Austin
61e738cd84 use enumValue instead of static_cast<int>
Summary:
Where appropriate, replace uses of `static_cast<int>` with
`enumValue`.

Reviewed By: simpkins

Differential Revision: D20975196

fbshipit-source-id: 581643366ea7eda5d1961238b0693cf45c4eec94
2020-04-28 18:59:34 -07:00
Chad Austin
daaeb5012a replace some uses of memset
Summary: Easier to zero initialize structs with braces, which defines that even padding is zeroed.

Reviewed By: wez

Differential Revision: D19655675

fbshipit-source-id: 2fd12383324029646707e93008cf9ad34e9f1dce
2020-01-31 10:50:48 -08:00
Zeyi (Rice) Fan
ed9e7abca2 eden: add FakeEdenConfig for testing
Summary:
I think we need to make `EdenConfig` available for testing as more things are using `EdenConfig` to configure behaviors.

Right now it does not do much things other than just construct an `EdenConfig` instance without manually providing these parameters.

Reviewed By: chadaustin

Differential Revision: D18798484

fbshipit-source-id: b4a85d966a9b4f425c547bef9eb6e7570d7e2170
2019-12-10 13:40:55 -08:00
Chad Austin
f930a40434 make PathComponentPiece and RelativePathPiece constexpr
Summary: Allow creation of PathComponentPiece and RelativePathPiece values at compile-time.

Reviewed By: wez

Differential Revision: D18642594

fbshipit-source-id: 209e5c27e8fab1e877ccee8558fa757b68078e66
2019-12-04 13:30:18 -08:00
Chad Austin
714e96f8aa config refactoring
Summary: While reading the config code, I made some minor, behavior-changing improvements.

Reviewed By: genevievehelsel

Differential Revision: D17918029

fbshipit-source-id: e40bce099a9555559c028c701f6fb75a6a4fc6be
2019-10-14 19:18:04 -07:00
Chad Austin
4b47257165 remove EdenConfig accessors
Summary:
Instead of having accessors for every config setting in EdenConfig,
just expose the ConfigSettings directly.

Reviewed By: fanzeyi

Differential Revision: D17847805

fbshipit-source-id: 8c6c1010c010113cf859677449797ea916f2a2a5
2019-10-11 17:55:19 -07:00
Andres Suarez
fbdb46f5cb Tidy up license headers
Reviewed By: chadaustin

Differential Revision: D17872966

fbshipit-source-id: cd60a364a2146f0dadbeca693b1d4a5d7c97ff63
2019-10-11 05:28:23 -07:00
Puneet Kaushik
92f45a1b0c Check if the fd is valid before calling close
Summary: This code calls close on fd while exiting. If the file doesn't exist the open will return -1 and close (-1) will be called.

Reviewed By: strager

Differential Revision: D15951485

fbshipit-source-id: ea3a52517847d75e9a822e51f360be7cb2c411da
2019-06-21 17:05:07 -07:00
Adam Simpkins
aa5e6c7295 update license headers in C++ files
Summary:
Update the copyright & license headers in C++ files to reflect the
relicensing to GPLv2

Reviewed By: wez

Differential Revision: D15487078

fbshipit-source-id: 19f24c933a64ecad0d3a692d0f8d2a38b4194b1d
2019-06-19 17:02:45 -07:00
Adam Simpkins
7309869981 add a thrift call for getting config values
Summary:
Add a thrift call to get the current config settings.

My primary use case for this method at the moment is to make it possible to
build integration tests that check the config behavior.  However in the future
this will probably also be useful for building CLI commands to report the
current config values to allow debugging if there are ever issues.  This API
can also be used to force EdenFS to immediately reload the config from disk.

Reviewed By: strager

Differential Revision: D15572124

fbshipit-source-id: da3bc982f9c419b3314a8b0560c9bd327760d429
2019-06-11 13:08:28 -07:00
Adam Simpkins
897764d81c move the ConfigSource enum to a thrift file
Summary:
Move the ConfigSource enum definition to a thrift file.  This will let us
return ConfigSource values over thrift APIs in the future.  This also allows
us to use thrift's `TEnumTraits` functionality to determine the maximum enum
value, rather than having to maintain a separate `kConfigSourceLastIndex`
variable.

As part of this change I also renamed the enum values to be CamelCase to match
our current C++ style recommendations and to avoid possibly conflicting with
macros defined in other headers (`DEFAULT` seemed particularly susceptible to
collision).

Reviewed By: strager

Differential Revision: D15572120

fbshipit-source-id: 8fbd03da221a9f75ef670dee1eb250eb198a5bd0
2019-06-05 11:50:37 -07:00
Puneet Kaushik
9e24d970a8 Replace EDEN_WIN with _WIN32 for Windows only code
Reviewed By: chadaustin

Differential Revision: D15228464

fbshipit-source-id: 08b30f535fa17a5bcda906e640cc036cd0d59d02
2019-05-09 16:33:08 -07:00
Puneet Kaushik
d991894164 Moving Edenfs's Windows code to eden/fs
Reviewed By: wez

Differential Revision: D15224021

fbshipit-source-id: d046da829a6bceb73c7f05c62a77a0ef9a39e4bc
2019-05-09 16:33:08 -07:00
Zeyi (Rice) Fan
b48f1db479 copy userName_ and userID_ during EdenConfig copy assignment
Summary:
This solves the client certificate mystery. We creates a new `EdenConfig` object
whenever there is a change detected and these two fields are lost during the
copying. Causing the variable substitution not working as expected.

Reviewed By: wez

Differential Revision: D15221180

fbshipit-source-id: 3fba36c8f261cb3dbe25e2e363d65661240be8cf
2019-05-08 16:18:32 -07:00
Adam Simpkins
1ce08e9727 turn ConfigSource into an enum class
Summary: Change `ConfigSource` from a plain `enum` to an `enum class`.

Reviewed By: chadaustin

Differential Revision: D15162811

fbshipit-source-id: 680ec1e785de1a54b623d99477914afa5d0931ca
2019-05-08 12:03:21 -07:00
Adam Simpkins
b4ca002c0c move the FieldConverter code to its own file
Summary:
Move the `FieldConverter` logic out of EdenConfig.h/cpp and into its own
module.

Reviewed By: chadaustin

Differential Revision: D15162816

fbshipit-source-id: f30d81329139e5827b81a609e83baddf71b6af5e
2019-05-07 17:41:36 -07:00
Matt Glazar
65f80da385 Add USER_ID variable for config options
Summary:
The systemctl command requires XDG_RUNTIME_DIR to be configured. If it's not configured, 'eden start' should pick a sane default. The sane default includes the user's UID (e.g. /run/user/6986). I want this default to be configurable via Eden's config files.

Expose the ${USER_ID} token to Eden configs. This will let administrators can customize XDG_RUNTIME_DIR's fallback value in the future.

Reviewed By: wez

Differential Revision: D13811732

fbshipit-source-id: 7933e078dd5f2b3bbbb0299730220a129c257256
2019-03-14 18:19:41 -07:00
Zeyi Fan
d9f1beb1a1 add mononoke:connector config and create initializeHttpMononokeBackingStore
Summary:
This diff added a configurable value "mononoke:connector" to EdenConfig, so that we can specify a particular type of connection we need when using Mononoke as backing store.

This diff also moved the logic of initializing the existing `MononokeHttpBackingStore` to `initializeHttpMononokeBackingStore`.

Reviewed By: wez, strager

Differential Revision: D14141725

fbshipit-source-id: c7208295d7b3853740d7b0e5166f8854457fcf8e
2019-03-11 14:34:08 -07:00
Chad Austin
28c480cb03 use cpptoml from third-party
Summary: Switch to using cpptoml from third-party on all platforms.

Reviewed By: igorsugak

Differential Revision: D14351179

fbshipit-source-id: 183bedc7d27e9c0f9216f3d0cca58ada75ac74e7
2019-03-06 15:19:23 -08:00
Adam Simpkins
6d44b3e88c change the default ignore file path to ~/.edenignore
Summary:
Previously the default ignore path was `~/ignore`.  This doesn't really seem
like a great path choice, and I suspect that no one is actually using an
ignore file at this location.  This default was set in D8594297, but I it
looks like this was mostly accidental from just not separating out data for
the system ignore path (`/etc/eden/ignore`) vs the user ignore path.

Using `~/.edenignore` seems reasonable for now, and unlikely to conflict with
other paths.  We used to use `~/.gitignore` as the default before D8906226,
but this did cause problems for a handful of users that treated their entire
home directory as a git repository, and had a `~/.gitignore` file that was
supposed to apply only to their home directory.

Reviewed By: wez

Differential Revision: D13984153

fbshipit-source-id: 887528372b9be789317933f7026dfcbde8cd4539
2019-02-07 18:31:45 -08:00
Puneet Kaushik
ea61afec34 Create a platform independent way to compare the stats.
Summary: Create a platform independent function to compare the stats and use that to check if the file contents have changed.

Reviewed By: strager

Differential Revision: D13046269

fbshipit-source-id: c4f5bc88cec3df5cb6555d13cea2f23fd4eeb7ce
2018-11-21 16:38:51 -08:00
Wez Furlong
1543b871d7 expand multiple ${USER} tokens when processing config
Summary:
It is desirable to be able to reference the same variable
multiple times in the RHS of a config setting.  This diff makes that
possible.

Reviewed By: strager

Differential Revision: D12906500

fbshipit-source-id: 4277f12105d0a0fb3dca880d3dad6b0238acedc0
2018-11-09 16:19:50 -08:00
Wez Furlong
19d6dd41d9 add pathFuncs expandUser() for tilde expansion
Summary:
This function behaves similarly to the python `os.path.expanduser`
function, except that it is restricted to expanding only information for the
current user.

Use this new function to process the hgcache path returned via the
rust config parsing code.

In the longer term I want to centralize and add accessors for
the rust config and move this stuff into EdenConfig, but for
now this is the one place that does this and it is OK to
process it this way here.

Reviewed By: strager

Differential Revision: D12988902

fbshipit-source-id: 96b10640359c3b8c0adac1e14cd42dd592023c3d
2018-11-09 11:24:37 -08:00
Wez Furlong
24f0b35903 add option to override the mononoke tier name
Summary:
To facilitate testing against a test instance rather than selecting
from a tier, add a config option to return the host override.

The semantics are that a hostname string will override the tier name setting.
An empty host name string is treated as missing and will not replace the
tier name.

Reviewed By: chadaustin

Differential Revision: D12813050

fbshipit-source-id: 67773d772d78d3f7882825ce54f4313c97ab0a33
2018-10-31 11:50:39 -07:00
Chad Austin
d5a0a0d063 folly::Optional -> std::optional
Summary: Eden's on C++17 so fully cross the rubicon!

Reviewed By: strager

Differential Revision: D10496796

fbshipit-source-id: 3b0b8e10a93830c0e76dbc5baf95cf69a1f080af
2018-10-23 17:05:11 -07:00
Puneet Kaushik
1e94122c0b Win: Porting eden/service to Windows with fb-thrift
Reviewed By: strager

Differential Revision: D10225935

fbshipit-source-id: 5c571229e243b8fcabb9db8a6e15e1c66423edaf
2018-10-22 20:27:26 -07:00
Wez Furlong
da904dd4a5 make mononoke tier configurable
Summary: This was previously hard coded and making it configurable is operationally desirable.

Reviewed By: chadaustin

Differential Revision: D10298279

fbshipit-source-id: 124c3fa319c44aacb3d17ef6e380e9dec49db8be
2018-10-10 14:47:08 -07:00
Wez Furlong
35b902fdc2 fix toml config parsing issue with booleans
Summary:
Unfortunately, our current config parsing implementation is
forcing us to stringly-type all data, which partially defeats one
of the benefits of using a modern typed configuration file format
such as TOML.

This means that boolean options have to be specified as strings in
the config file.

I don't want to ship the Mononoke feature integration out and tell
folks to use `"true"` to turn it on as it will impose a future migration
concern on us.

This diff does the simple but gross thing: at parse time, rather than
ignoring boolean values, we convert them to string so that the upper
layer can convert them to bools again.

Reviewed By: strager

Differential Revision: D9582582

fbshipit-source-id: a8eb8a8e4b59e3d74b3371b743a6eb590b3b0f58
2018-08-31 13:07:52 -07:00
Zeyi Fan
96c7bb1c5f read use-mononoke and client-certificate in edenfs
Summary: This commit let `HgImporter` pick up configuration values from `EdenConfig`.

Reviewed By: chadaustin

Differential Revision: D9346293

fbshipit-source-id: cb63f7d13a86058e9bf076eddb52212560a64cb1
2018-08-20 11:22:14 -07:00
Zeyi Fan
10dfcf8da4 add two new configuration value for Mononoke
Summary: This commit adds two configurable value for setting path to client certificate and a flag to control Mononoke integration.

Reviewed By: chadaustin

Differential Revision: D9303157

fbshipit-source-id: 2f44d55d17b567655157a5f4b6f52e9468dda234
2018-08-15 10:51:44 -07:00
Eamonn Kent
b857159384 Allow USER and HOME to be interpolated in Eden c++ code
Summary:
The python code supports expansion of ${HOME} and ${USER} in the config files.
Some of the config files use these values. This adds support for their
interpolation.

Reviewed By: chadaustin

Differential Revision: D9160960

fbshipit-source-id: 9133ee247c17937c9d8d548b9bfd26559a1459e9
2018-08-06 13:53:07 -07:00
Eamonn Kent
a5591d7a31 consistent naming for userIgnoreFile
Summary:
Let's use consistent naming for the user ignore file. We will go with
'userIgnoreFile' as recomended. There is now symetry since we have
'systemIgnoreFile'.

I have another diff (later) that already uses the 'userIgnoreFile'.

Reviewed By: chadaustin

Differential Revision: D8823906

fbshipit-source-id: 47594e8971fa4db809821fc819da05e9eb2c5277
2018-07-13 10:51:08 -07:00
Eamonn Kent
bb9c1817de Improve file change detection precision in EdenConfig
Summary:
EdenConfig uses stat to determine if config files have change. However, it
should use st_mtim.tv_nsec rather than st_mtime because this provides
nano-second resolution.

Reviewed By: simpkins

Differential Revision: D8797143

fbshipit-source-id: aadf114f509116d497a4c237ecaf108eb3824d5f
2018-07-11 15:06:31 -07:00
Eamonn Kent
601c5ea1e5 Support loading of system wide and users eden config files
Summary:
This commit focuses on the following functionality:
    - Eden configuration should have reasonable default values.
    - Eden should allow configuration to be specified in various manners: default, cli, user and system configuration files.
    - Configuration values should have the following precedence (hi to low): cli,
      user config, system config, default.
    - A light-weight mechanism is required to detect and update configuration changes.

Reviewed By: simpkins

Differential Revision: D8594297

fbshipit-source-id: ed195212669b18b450a1eae359e4d23905beadb4
2018-07-09 16:54:11 -07:00