Summary:
There is a race in here. We read the config, then mutate it, then put the new
config in place. However, if something else comes and mutates the config
between the time where we read it and write it again, we'll clobber its
changes.
Concretely, if you're running lots of `eden clone` or `eden rm` concurrently,
then the current mechanism will sometimes "forget" some directories or add back
directories you removed, which will e.g. break `eden list` (and cause those
commands to sometimes fail as well).
This diff fixes that by wrapping the critical section with a file lock.
Reviewed By: chadaustin, xavierd
Differential Revision: D32251727
fbshipit-source-id: 9eee57c7a824286bb4f0e7911e5ccd9d769c6daa
Summary:
This config has been enabled for many months now, let's simply assume it is set
to true on Windows.
Reviewed By: genevievehelsel
Differential Revision: D32192585
fbshipit-source-id: 76829a031f859594a5139dd09a6479cabeda2089
Summary: When debugging, it's nice to know what the kernel thinks the mount table is.
Reviewed By: xavierd
Differential Revision: D32288215
fbshipit-source-id: d4c15b16689f20846a45f682c4089e2fa8250661
Summary:
Since the CWD wasn't specified when gathering the Buck version, the buck
version returned by `buck --version-fast` could be different from the one in
use in the subdirectory where `buck kill` would be called. This would lead to
Buck failing to stop itself, which would interrupt `eden stop`.
Reviewed By: kmancini
Differential Revision: D32193032
fbshipit-source-id: cbe4983df785b4439645037f9d42084cf745a423
Summary:
The original bug has been fixed for over a year now, let's remove the
workaround.
Reviewed By: kmancini
Differential Revision: D32193033
fbshipit-source-id: 29de7a52e47f8fe070b45b785756b4c93e42151c
Summary:
C++20 changed the rules around char8_t. Cast to char* as
necessary. Also, fix references to some removed std::allocator
associated types.
Reviewed By: fanzeyi, meyering
Differential Revision: D32155762
fbshipit-source-id: dc72e5bc58e8c08612ea1a1eb2568861701287c6
Summary:
When running the Eden tests using buck2, this flag is used because the
privhelper is not immediately next to the EdenFS binary.
This diff updates the approximation of the privhelper path introduced in
D31870880 (710f03501d) accordingly so that it also works in this case.
Reviewed By: xavierd
Differential Revision: D32070038
fbshipit-source-id: c111cc533bb9e9d111bfe9bad71e817655b3d613
Summary:
In order to execute the edenfs_privhelper, EdenFS needs to be a suid root
binary, but due to the large amount of dependency, it's hard to fully control
what runs before main and thus what executes with root privileges.
To reduce the attack surface, let's only make the edenfs_privhelper a suid root
binary. In tests, we still start EdenFS with sudo, and thus the
edenfs_privhelper will have the proper privileges.
Reviewed By: chadaustin
Differential Revision: D31870880
fbshipit-source-id: 3f06d22ecd4f3bc65e3e955c5a1de811b1cf328f
Summary:
As a first step, as I ramp up on Rust and the edenfsctl codebase and in order to more quickly support basic top functionality in windows, we're adding a minitop subcommand that will have a subset of the `eden top` output.
This diff adds scaffolding for the minitop subcommand.
Reviewed By: fanzeyi
Differential Revision: D31841478
fbshipit-source-id: 7020e81a0c8021bad45de835f2e279c049b76200
Summary:
When eden is not running on windows `eden rm` will spit out some error
messages and exits with error code 1. But it does actually successfully
remove the repo.
On linux removing a repo while eden is not running behaves just like if
eden were running.
Let's make the removal more graceful on windows.
Reviewed By: xavierd
Differential Revision: D31519805
fbshipit-source-id: d393922a9474e64251142207ae38a602766f17bf
Summary:
xterm-color terminals don't support hiding the cursor the way curses likes to.
this crashes eden top for users.
eden top mostly functions fine otherwise in xterm-color, so let's make
this a non fatal error.
Note: the highlighting behavior is a little wierd (the styling after the
highlighted column does not work, no bolding etc). This looks like a wierd
curses bug on these terminals, so not sure if we can really fix this. I figure
a little wierd styling is still better than a crash.
Reviewed By: fanzeyi
Differential Revision: D31480121
fbshipit-source-id: 581ef7c548fd1f7986c4f93d8b797d7f7588c351
Summary: This diff adds the Rust edenfsctl to our open source build.
Reviewed By: xavierd
Differential Revision: D30788685
fbshipit-source-id: 603caa933ecfc5af0ede7e22f6c7911462da3a65
Summary:
Now that we might have multiple kernel protocols per mount (i.e. both fuse and
nfs on macOS) let's include them in eden rage.
Reviewed By: xavierd
Differential Revision: D31154042
fbshipit-source-id: 38e7630829d70fe9dd6dbeabacc3b538ee798e0d
Summary:
we are passing some bytes into Popen and shlex.quote. shlex.quote expects a
string not bytes. fsencode gives us bytes fsdecode gives us string. Let's used
fsdecode instead.
Reviewed By: zhengchaol
Differential Revision: D31129335
fbshipit-source-id: 7792bdcd4dd833a4946daf8ec75576cfe4fc24af
Summary:
During an `eden chown`, EdenFS will try to chown both the repository, and the
redirections. In some cases, chowning the redirection can both take a long time
and be unecessary. Consider the case where some automation temporarily chown a
repository to a service user that needs to access the repository, and then
chown it back to the owner of the repository. In that case, changing the
ownership of the redirection is superfluous and unecessary.
Reviewed By: mrkmndz
Differential Revision: D31010912
fbshipit-source-id: a882948005ac4fe29ff465088f196e0fc2bc10be
Summary:
In D29940980 (2e2b9755cf) we used shlex for a redirect subprocess command line.
The list does not always contain strings tho, which is a requirement to use
shlex.quote my guess is that they are paths. We should still str things
before we shlex.quote them.
Differential Revision: D31001622
fbshipit-source-id: 2a270781d7f2d84ad7a9a2f9975500b29306cfa8
Summary:
We have an option on GlobFiles for listing hidden files, but we don't have a
cli option. We default to false in the cli. Let's pipe this option all the way through.
so that we can control this flag from the cli.
Reviewed By: xavierd
Differential Revision: D30915118
fbshipit-source-id: 28b91d4fd2dd4bdf9e342929f570f64db14e8ad0
Summary:
`eden prefetch` and `eden glob` return lists that despite being called
"maching files" actually contains both files and directories.
In some cases, we only want the list of files and it introduces unnessecary
overhead on our clients for them to have to stat all the files in the list to
filter out the dirs. Let's add an option to just list files.
Reviewed By: chadaustin
Differential Revision: D30816193
fbshipit-source-id: 6e264142162ce03e560c969a0c0dbbc2f418d7b9
Summary: the rage summary is getting hard to quickly parse, so this underlines each section header, as well as unifies underline looks (with `eden stats`). This adopts the underline code from `eden du` and makes it a util function for shareability.
Differential Revision: D30857773
fbshipit-source-id: 66b5b06f5b0125304d45d3465a8bc2248693b791
Summary: it would be helpful to see a user's or sandcastle job's eden config, especially in the case of a gated feature rollout / staged feature rollout.
Differential Revision: D30857763
fbshipit-source-id: ee2a311ee643fc9db5acef1b02017564c51d2362
Summary:
just a typo fix
Created from CodeHub with https://fburl.com/edit-in-codehub
Reviewed By: fanzeyi
Differential Revision: D30849172
fbshipit-source-id: 9779832870c909d080548ec71ecf86aa53767dbc
Summary: If the bind unmount fails in in the privhelper, theres a possibility of infinite recursion in this method. This adds a flag to indicate if we've tried the bind unmount before.
Differential Revision: D30732857
fbshipit-source-id: 6ee887d211977ee94c8e66531287f076a7e61a2c
Summary: Adds an option to print the path to the eden log file. Similar to `eden pid`, this can be used for shell one-liners.
Reviewed By: chadaustin
Differential Revision: D30558294
fbshipit-source-id: ca70addaef2093e10f0321bae0cff3b1bfc7dc75
Summary: `eden debug log --upload` fits in better with the format of the other cli tools (rather than `eden debug log upload`)
Differential Revision: D30557691
fbshipit-source-id: 32e47e1487703560f2adb5f0f79f1002d29eea93
Summary: Added a kill switch to enable/disable predictive prefetch profiles similar to the existing one for regular prefetch profiles (D24803728 (7dccb8a49f)). This can be set manually in a user's config or via the cli `eden prefetch-profile disable-predictive/enable-predictive` commands.
Reviewed By: genevievehelsel
Differential Revision: D30404139
fbshipit-source-id: 01900f4030ef6991124f89a67ea404ff2f07ffeb
Summary:
Added eden prefetch-profile activate-predictive/deactivate-predictive subcommands to activate and deactivate predictive prefetch profiles. This will update the checkout config to indicate if predictive prefetch profiles are currently active or not, and stores the overridden num_dirs if specified on activate (--num-dirs N). If activate is called twice with different num_dirs, the value is updated (only one is stored). Unless --skip-prefetch is specified, a predictive prefetch with num_dirs globs (or the default inferred in the daemon) is run.
Also added fetch-predictive [--num-dirs N], which will:
1. if num_dirs is specified: fetch num_dirs globs predictively
2. if num_dirs is not specified, and predictive fetch is active: get the active num_dirs from the checkout config and fetch globs predictively
3. if num_dirs is not specified, and predictive fetch is not active: fetch the default num_dirs (inferred in the daemon)
Added --if-active to fetch-predictive. If set, fetch will not run if predictive prefetch profiles have not been activated (predictive-prefetch-active in checkout-config). Used for post pull hook.
Reviewed By: genevievehelsel
Differential Revision: D30306235
fbshipit-source-id: ba02c2bc976128704c8ab0c3d567637265b7c95d
Summary: This is an untended pop and would throw if there is no stale apfs volumes (and would remove one less volume if there are stale volumes).
Reviewed By: xavierd
Differential Revision: D30432642
fbshipit-source-id: 193d9c15f393a66bc8b43b5f31579c1fe972a7f1
Summary:
This adds the `eden redirect cleanup-apfs` command for deleting stale APFS volumes.
* An APFS is considered as stale if it's not currently mounted and not considered as under any of the checkouts managed by the eden instance.
* The command prints the list of such volumes and uses the APFS util to delete them if the user confirms.
Note: as the command is local to an eden instance, it will list not-mounted APFS volumes of a checkout managed by another eden instance as stale. This should rarely happen as in production we expect there to be a single eden instance. The prompt would also let the user abort if something is wrong.
Reviewed By: chadaustin
Differential Revision: D29940980
fbshipit-source-id: e784cb54d20198bb1f74cd5f15cee0e7546b227c
Summary: `edenfsctl rm` often breaks down due to long path on Windows. This diff fixes that issue.
Reviewed By: xavierd
Differential Revision: D30380003
fbshipit-source-id: e10faa357df932bdb49d7c62d04d9504c7885768
Summary: This diff teaches eden doctor to generate a warning when user has SQLite overlay repo on disk and asking them to migrate.
Reviewed By: chadaustin
Differential Revision: D30345721
fbshipit-source-id: 95796ca77979f034904b87e3a38f149baddd720a
Summary:
When a user reports a slow EdenFS and a high network traffic is suspected, the
lack of Thrift stats makes it hard to fully validate this. Thus, let's collect
some stats and put them in the rage.
The collected stats are the exact same ones that `eden top` uses.
Reviewed By: chadaustin
Differential Revision: D30355746
fbshipit-source-id: 519a8e2c8b0c458daecdcc0813a8d7416d5362d6
Summary:
This newly added config controls the default behavior of `eden clone` and
whether a new clone will use NFS or FUSE. This is intended to facilitate the
transition to NFS from FUSE on macOS.
Differential Revision: D30110056
fbshipit-source-id: ea6b493aa803297952b46434f6d11d8edf58e40b
Summary: As title. This was added as `thrift-py3` doesn't support CMake well but we can workaround by importing from py.
Reviewed By: xavierd
Differential Revision: D30078300
fbshipit-source-id: 277866f8b226f164b5e30231aa10b59c0aba5807
Summary:
This adds the options to `eden stats` for collecting only fast stats and printing in JSON.
`eden stats` can be slow especially due to collecting fb303 counters and private bytes. An example use case of this new lightweight endpoint is that Buck can poll it to display Eden related info in its cli (see [post](https://fb.workplace.com/groups/132499338763090/permalink/210396380973385/) for context).
Reviewed By: xavierd
Differential Revision: D29687041
fbshipit-source-id: a663e71231527c5dfb822acbf238af0ac6ce4a00
Summary: Currently we have to manually save the id returned from `start_recording`. After this, we can simply ask for the list of all active recorder sessions.
Reviewed By: genevievehelsel
Differential Revision: D30056117
fbshipit-source-id: 7fd69b70e7b04fcd0b3724f4ee16c5e5e86badaf
Summary:
Per comment in D30017261 (e9c039ab4a)
> [...] we can get rid of this ESTALE check. That dates back to when edenfs had a bug that returned ESTALE when reading the .eden directory sometimes.
Reviewed By: xavierd
Differential Revision: D30024979
fbshipit-source-id: 645097c8f689c916245845561fd3d824ff7df8b4
Summary: `eden mount` currently swallows all but one type of exceptions, which resulted in cli reporting false positive mount success when the thrift endpoint throws. The try-catch was added in D21934538 (51df752a46) to provide clear error message for a particular windows error and we can just re-raise so that the other exceptions are propagated.
Reviewed By: chadaustin
Differential Revision: D30017261
fbshipit-source-id: 6a8a44330a90275b3c044301ff644dce0d6dee13
Summary:
Today the globing code is not able to parse `\` as a path separator
which causes an error like this:
```
facebook.eden.ttypes.EdenError: Invalid glob (attempt to construct a PathComponent from a string containing a directory separator: tools\arcanist): tools\arcanist
```
Many other tools like hg files still use these separators, so it would be
nice if we could support `\`.
This change will mean that Windows globs will not support special characters.
But they were already broken so we can not have any users successfully using
them today.
Ideally we will not have to support special charcters on Windows because it is
icky, But I left a note in the code about how we could do this, if its ever
needed in the future
Reviewed By: chadaustin
Differential Revision: D29971586
fbshipit-source-id: 32f54986e801b2aa3b3b1c90c650bd45531e8c0a
Summary:
If Mercurial asks EdenFS to update to a commit that it has just created, this
can cause a long delay while EdenFS tries to import the commit.
EdenFS needs to resolve the commit to a root manifest. It does this via the
import helper, but the import helper won't know about the commit until it is
restarted, which takes a long time.
To fix this, we add an optional "root manifest" parameter to the checkout or
reset parents thrift calls. This allows the Mercurial client to inform EdenFS
of the root manifest that it already knows about, allowing EdenFS to skip this
step.
Reviewed By: chadaustin
Differential Revision: D29845604
fbshipit-source-id: 61736d84971cd2dd9a8fdaa29a1578386246e4bf
