Summary:
Per comment in D30017261 (e9c039ab4a)
> [...] we can get rid of this ESTALE check. That dates back to when edenfs had a bug that returned ESTALE when reading the .eden directory sometimes.
Reviewed By: xavierd
Differential Revision: D30024979
fbshipit-source-id: 645097c8f689c916245845561fd3d824ff7df8b4
Summary: `eden mount` currently swallows all but one type of exceptions, which resulted in cli reporting false positive mount success when the thrift endpoint throws. The try-catch was added in D21934538 (51df752a46) to provide clear error message for a particular windows error and we can just re-raise so that the other exceptions are propagated.
Reviewed By: chadaustin
Differential Revision: D30017261
fbshipit-source-id: 6a8a44330a90275b3c044301ff644dce0d6dee13
Summary: This change let Eden cli can ```clone``` and ```info``` on a RE Digest backed store
Reviewed By: chadaustin
Differential Revision: D28855458
fbshipit-source-id: 5582992acc5b3b3acb05b0b53d59a6768cc02491
Summary: Extended eden doctor to check if the PrivHelper is accessible and report when it is not.
Reviewed By: genevievehelsel
Differential Revision: D29593250
fbshipit-source-id: 2390e75b91c9d6f713db4b6084868af91a0b6623
Summary: There are a lot of places in user visible text such as command line help where EdenFS is mentioned as eden/Eden/edenfs/EdenFS. This will make it consistent to 'EdenFS' in most cases. In the places where it is referring to the process/daemon, 'edenfs' will be used.
Reviewed By: chadaustin
Differential Revision: D29419151
fbshipit-source-id: 7b8296f0a0c84fdcb566ff811f7fcedbe7079189
Summary:
The enable-tree-overlay is not a top-level config, and thus this code would
never read a boolean value, thus when the config would be read, the
enable_tree_overlay would always be false.
Reviewed By: fanzeyi
Differential Revision: D29494144
fbshipit-source-id: 1537f3e12bf043e154b81e4c0672abf80d54e16c
Summary:
Add support to our C++ and Python SNAPSHOT parsers for version 2,
which is just an arbitrary binary string after the header.
Don't start writing the format yet. I'd like to roll one release with
read support before writing the new format.
Reviewed By: xavierd
Differential Revision: D28528235
fbshipit-source-id: 0b06596f06cafa26258ab392eaae4a8994f55f1f
Summary:
This fixes the handful of errors and warnings that Pyre threw when I tried
making this file strict. Enabling pyre strict will force future addition and
modifications to this file to have type checking, increasing the reliability of
the code.
Reviewed By: fanzeyi
Differential Revision: D28543212
fbshipit-source-id: 06b976a37e6f4c3a0bab2eb3ecc4c217497bbbd5
Summary: This diff makes treeoverlay the default overlay type for Windows users.
Reviewed By: kmancini
Differential Revision: D27247658
fbshipit-source-id: 866eafc794eff1c262eab3061f14eb597bea0a66
Summary: adds a new subcommand `disable` which will update the user's local eden config with a flag to disable prefetching
Reviewed By: kmancini
Differential Revision: D26964514
fbshipit-source-id: 043c89a5848f827377744ebc46fe93c431efee3e
Summary:
The world has moved on utf-8 as the default encoding for files and data, but
EdenFS still accepts non utf-8 filenames to be written to it. In fact, most of
the time when a non utf-8 file is written to the working copy, and even though
EdenFS handles it properly, Mercurial ends up freaking out and crash. In all of
these cases, non-utf8 files were not intentional, and thus refusing to create
them wouldn't be a loss of functionality.
Note that this diff makes the asumption that Mercurial's manifest only accept
utf8 path, and thus we only have to protect against files being created in the
working copy that aren't utf8.
The unfortunate part of this diff is that it makes importing trees a bit more
expensive as testing that a path is utf8 valid is not free.
Reviewed By: chadaustin
Differential Revision: D25442975
fbshipit-source-id: 89341a004272736a61639751da43c2e9c673d5b3
Summary:
The default filesystem on macOS (APFS) is case-insensitive, but EdenFS has so
far been case-sensitive except on Windows. Some of the native tools (Unity for
instance), expect macOS to always be case-insensitive, and is thus breaking due
to that.
The safe behavior would be to have EdenFS behave exactly the same as APFS: be
case insensitive. For now, to avoid breaking users this will be done on new
mounts only, and once fully validated, this will be made the default and forced
on.
Reviewed By: chadaustin
Differential Revision: D26356269
fbshipit-source-id: 96ca331d8c9726213520dff3e3563019d0400a95
Summary: This enables the repository to be mounted via NFS, and not FUSE.
Reviewed By: chadaustin
Differential Revision: D26229827
fbshipit-source-id: 5af5a47ebe5f1dd54df7707bf57d9b7476921f29
Summary:
As described in D26033704, it would be really nice if we could have a version of `eden rm` that did everything except remove the mount point directory. This creates a nice symmetry between the tolerated start and end points of a clone/rm cycle, in that `eden clone` already tolerates being targeted against and empty directory.
NOTE: I made no effort to make this work on Windows. The implementation was different enough that I wasn't exactly sure how you all wanted to go about it. We don't run on Windows at all, so this is all we need, but I'd understand if you all want to make it work, or at least throw an explanatory error.
Reviewed By: chadaustin
Differential Revision: D26033912
fbshipit-source-id: 36d63ad510393d3d09b1912a73efaebede534f61
Summary:
A lot of users have been having trouble getting their eden repos recloned
recently, to make this process simplier I add the reclone process to the clone
script.
In this script I check if a user has multiple repos depending on the same
backing store. This allows me to warn them that they might lose changes from
these other repos. This diff threads along the backing store to the
`eden list` result for that check.
Reviewed By: chadaustin
Differential Revision: D25078423
fbshipit-source-id: 9ceb1f9acc4ec170cbb12d4b0b3b7d51987f88e3
Summary:
On Windows, it's possible that not all files could be removed from the
repository due to some other process holding a reference to it. When that
happens the `edenfsctl rm` operation will fail. Sometimes, instead of failing
with the actual reason for why the removal failed, it throws a cryptic "list
index out of range" error.
The reason was that when the file that can't be removed is actually a
directory, the `errors` list would be empty. Since filtering the folders is a
bit silly, let's not do it.
Reviewed By: fanzeyi
Differential Revision: D25836412
fbshipit-source-id: 36f936ff9d7697dfd2f4c68d4e56bdb18b66b06a
Summary:
On OSX, if Mercurial is built from fbcode, these environment variables
(which point specifically to Eden's own par file data) can break Mercurial's
ability to load dynamic libraries. Let's unset them.
Reviewed By: xavierd
Differential Revision: D25783552
fbshipit-source-id: 74e6232d225856fedc0382abc6cd223a6c47d8bc
Summary:
On Windows, the GUID of the mount point identifies the virtualization instance,
that GUID is then propagated automatically to the created placeholders when
these are created as a response to a getPlaceholderInfo callback.
When the placeholders are created by EdenFS when invalidating directories we
have to pass GUID. The documentation isn't clear about whether that GUID needs
to be identical to the mount point GUID, but for a very long time these have
been mismatching due to the mount point GUID being generated at startup time
and not re-used.
One of the most common issue that users have reported is that sometimes
operations on the repository start failing with the error "The provider that
supports file system virtualization is temporarily unavailable". Looking at the
output of `fsutil reparsepoint query` for all the directories from the file
that triggers the error to the root of the repositories, shows that one of the
folder and its descendant don't share the same GUID, removing it solves the
issue.
It's not clear to me why this issue doesn't always reproduce when restarting
EdenFS, but a simple step that we can take to solve this is to always re-use
the GUID, and that hopefully will lead to the GUID always being the same and
the error to go away.
Reviewed By: fanzeyi
Differential Revision: D25513122
fbshipit-source-id: 0058dedbd7fd8ccae1c9527612ac220bc6775c69
Summary: In addition to tracking profile activation, it would be nice to see when people deactivate profiles.
Reviewed By: kmancini
Differential Revision: D25372256
fbshipit-source-id: e6bfefe3c838465ba9ed10d622342bdf3df1c40a
Summary: It would be interesting to track which profiles are activated most often. Also, it may be helpful to track which users are using this feature and generally track usage.
Reviewed By: kmancini
Differential Revision: D25372255
fbshipit-source-id: 2107cdce2a997180547dc63ec654c8d72ced5eb4
Summary: When I added `eden uptime`, I did not see that we already had `eden debug uptime`! Instead of having two, lets just use the one and remove it from debug. There seems to be a regression of the uptime I had implemented, so thats why I opted to use the implementation of the debug version.
Reviewed By: kmancini
Differential Revision: D24566844
fbshipit-source-id: d948a5303d475a543f51abbaea59f9c481dfeca2
Summary:
Add an `eden strace` command that subscribes to all FUSE events for
the specified mount (or current directory), and streams them to stdout
with response latency measurements and the requesting process.
If there are any pending requests at the time `eden strace` is run,
they are printed first.
Reviewed By: wez
Differential Revision: D24038978
fbshipit-source-id: 59a2112dbdb1708571d12b04bdccaf9eca725cf7
Summary:
Chef writes root-owned temporary files in /etc/eden/config.d. If
edenfsctl attempts to read those configs at this time, it will
fail. Just skip all hidden files.
Reviewed By: kmancini
Differential Revision: D24377953
fbshipit-source-id: 765b7f096b62e6749e186adc3e90bbc7814079c8
Summary:
This will allow us (and maybe users) to remove a users active profiles
easily when investigating issues. (And also will be useful for testing.)
Reviewed By: genevievehelsel
Differential Revision: D23517409
fbshipit-source-id: d57e595770b5a60726e2efe9cbf21c89dd694cc0
Summary:
This will allow tools to activate prefetch profiles relavant to their workflow.
Eventually this should also cause eden to prefetch the profile for the current
commit, but for now it just activates the profile.
Reviewed By: genevievehelsel
Differential Revision: D23517412
fbshipit-source-id: 172a844ca00bf3bcf14f4b94c25e8256e516b737
Summary:
This will allow us (and maybe users) to see a users active profiles
easily when investigating issues. (And also will be useful for testing.)
Reviewed By: genevievehelsel
Differential Revision: D23517410
fbshipit-source-id: 7a592f7b39c887600307a4f08668ca1ee9b982bf
Summary:
Assuming the format for active prefetch profiles as discussed here:
https://fb.quip.com/v8yFAzeGx8TU
This allows the cli to read in and write out active profiles, allowing the cli
to manipulate them in the following changes. This will allow tools to activate
profiles useful for that workflow. Additionally, it will allow us (and maybe
users) to see/remove a users active profiles easily when investigating issues.
Reviewed By: genevievehelsel
Differential Revision: D23517411
fbshipit-source-id: 8ab18a1aeccf8dcdd04aca826d61070fa0aedd33
Summary:
Rather than switch all of Eden at once to the thrift-py3 client,
rename get_thrift_client to get_thrift_client_legacy so uses of the
new client can be introduced piecemeal.
(I did try migrating everything at once but it's been quite painful.)
Reviewed By: fanzeyi
Differential Revision: D22423399
fbshipit-source-id: 9e6d938b90fff9fc3266ba20bc77e880e7f5b1aa
Summary:
There was a time when getRegexExportedValues was not in open source
fb303, but that time has long passed.
Reviewed By: kmancini
Differential Revision: D23745295
fbshipit-source-id: 4702068f0bb7350467e42439444b3f4d75aeec76
Summary:
`C:/tools/eden` will be overriden whenever a new EdenFS package is installed, therefore making it unsuitable to be managed by Chef.
Changing the default configuration directory to `C:\ProgramData\eden` that aligns with other programs.
Reviewed By: xavierd
Differential Revision: D23484626
fbshipit-source-id: 763518c608b24caa08e089a738f5c3577a0d6483
Summary: We don't need to restart users if their running version is the same as their installed version, so we should check that when deciding if we should restart. This will give us more freedom in restarts since we won't have to play with `min_uptime`. I will add a flag to skip this check in case for some reason we need to do so on the fly.
Reviewed By: wez
Differential Revision: D23438306
fbshipit-source-id: b17c0e13789071b8b7c1b15ac5a8deb74a4fd091
Summary: I want to be able to reverse engineer an EdenInstance in the `edenfs_restarter` given the cmdline of the process. I think this best lives in the `config.py` file.
Reviewed By: fanzeyi
Differential Revision: D23438318
fbshipit-source-id: b3d9ac3981d3fb2bb8045b07b8d949cd601f6898
Summary:
On Windows, unmounting the repo isn't sufficient to remove all traces of the
repo as ProjectedFS uses it as a cache, and thus all the files that were
touched by the user are still present on disk after unmounting it.
Reviewed By: wez
Differential Revision: D22915349
fbshipit-source-id: 445379fef57160b5da39a298ca4518e2662d16e3
Summary:
There was some missed usage of `Path.resolve`. This diff should cover it all.
```
cli $ rg -F ".resolve"
main.py
967: uid = self.resolve_uid(args.uid)
968: gid = self.resolve_gid(args.gid)
util.py
622: `Path.resolve`. This is a helper method to work around that by using
628: return path.resolve(strict=strict)
```
Reviewed By: chadaustin
Differential Revision: D22459188
fbshipit-source-id: c2a1b132f752cc399ebf34723f26123559939f2a
Summary:
On Windows, `pathlib.Path.resolve` has an use of uninitialized memory bug in Python 3.6.2. Lego Windows has this version of Python 3 installed.
When `eden prefetch` runs, it will first attempt to read the config file at the root of the repository (`$REPOROOT/.eden/config`). Because of this bug, when we normalize the path in edenfsctl, it will get some random bytes as a result. This subsequently causes `toml.read` to fail as it is unable to read a path containing NUL byte.
As you can see, this is the same exception as we see on Windows:
```
In [1]: open("test\x00")
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-1-57764b20d660> in <module>
----> 1 open("test\x00")
ValueError: embedded null byte
```
Reviewed By: xavierd
Differential Revision: D22177997
fbshipit-source-id: ab2565c8946d809bc15bc1597b749fb5e9440ca0
Summary:
On Windows, we can't rely on the .eden directory to not be present to decide if
the repository is mounted. Instead, we can just let edenfs decide and catch the
exception if it tells us that it's already mounted.
Reviewed By: fanzeyi
Differential Revision: D21934538
fbshipit-source-id: 60ed9f530456b627091f95f7387dc4b8f3a8dc5c
Summary:
When the scuba telemetry logger is unavailable, fall back on the
subprocess.
Reviewed By: genevievehelsel
Differential Revision: D21687855
fbshipit-source-id: c58a4bc1b34974add35d194c8aafdaac7ed47fdb
Summary:
The Python 2-and-3 Thrift API is sort of deprecated and does not
handle binary data in `binary` fields. In advance of migrating to the
modern Python 3 API, remane eden.thrift to eden.thrift.legacy.
Reviewed By: fanzeyi
Differential Revision: D21889697
fbshipit-source-id: a745ee8977999acbfb383a4edebe81d8deb1734e
Summary:
I am planning to start migrating Eden's CLI to the new Python 3 Thrift
implementation. In preparation, slightly clean up the interface and
implementation of our Python 2 Thrift wrapper.
Reviewed By: genevievehelsel
Differential Revision: D21854539
fbshipit-source-id: d398dd3f324c12288871cf0c9db41e64ed4cf7ed
Summary:
This diff is auto-generated to upgrade the Pyre version and suppress errors in eden. The upgrade will affect Pyre local configurations in the following directories:
```
eden
```
Differential Revision: D21687853
fbshipit-source-id: baf0d9bc33f86da63ea289690faca6cf4d566588
Summary:
Our use of the term "client" to refer to a checkout is
deprecated. Rename some internal functions that use the term client.
Reviewed By: simpkins
Differential Revision: D21395159
fbshipit-source-id: fa96ba593f53b493e5ae816fa686f333a132c232
Summary:
Addressing issues simpkins brought up on D21207287 when we upgraded and introduced some pyre bugs.
Temporarily upgrading just this project, once we resolve some sandcastle capacity issues we'll release this via another global upgrade in fbcode.
Reviewed By: simpkins
Differential Revision: D21316793
fbshipit-source-id: f0c79f53d97f7182e7d8fe6e081c58ef53ce0c9a
Summary:
On Windows, paths components are usually separated by '\', and since the
repository path is stored in a toml file, whatever character is after a '\',
will be escaped. In my case, this is followed by U (for C:\Users), and thus
toml expects the next characters to be an escaped unicode. That's obviously
not the case and thus EdenFS fails to parse the config, preventing me from
cloning fbsource.
Since Windows is perfectly fine with '/' as path separator, let's just
replace '\' with '/'.
The underlying bug appears to be in the toml Python code: https://github.com/uiri/toml/issues/280
Manually trying some random path is pretty conclusive:
(Pdb) toml.dumps({'foo': 'c:\\Users\\wez'})
'foo = "c:\\\\Users\\\\wez"\n'
(Pdb) toml.dumps({'foo': 'c:\\Users\\xavier'})
'foo = "c:\\Users\\xavier"\n'
Reviewed By: chadaustin
Differential Revision: D21143545
fbshipit-source-id: 448471da12c253dd37680f6a28251a1e69850920
