Summary:
It looks like 20s might not be sufficient anymore as the test is failing too
frequently with `wait` completing without the process having completed:
```
eden/fs/service/test/StartupLoggerTest.cpp:352: Failure
Expected equality of these values:
"exited with status 0"
returnCode.str()
Which is: "running"
eden/fs/service/test/StartupLoggerTest.cpp:363: Failure
Value of: isWritablePipeBroken(fd)
Actual: false
Expected: true
```
Reviewed By: kmancini
Differential Revision: D29342002
fbshipit-source-id: d457b632a2666356e9053d2da66a3227e6524b68
Summary:
In D29079762 (ea2e2f8bbd), globbing was fixed to not match the recursive glob (**) against
the entire path, as this would lead some paths to be matched while they
shouldn't. It however introduced another bug: in some cases, recursive globs
would no longer match paths that should be matched.
To fix both, a partial revert of the original diff is done with a small tweak:
the path that is matched against no longer starts at the root of the
repository. This will prevent `a/b/**/b/c.txt` to match `a/b/c.txt` as
`**/b/c.txt` would only be matched against `c.txt`, and not `a/b/c.txt` like it
was previously.
Reviewed By: fanzeyi
Differential Revision: D29175333
fbshipit-source-id: 1a4137d6f64f6cb77c4be09bd143f72630aa58d5
Summary:
When buck kill fails, eden rm will also fail. This has caused some checkouts
to not be removed when they could be. Stopping aux processes is a nice thing
to do before we unmount. It ensures these processes close file handles in the
mount, but we force unmount anyways so open file handles should not be able to
block the umount call.
Reviewed By: xavierd
Differential Revision: D29205962
fbshipit-source-id: a899940efa5cc1d960cd14a775b7053c34f5d6f2
Summary: This is more confusing than really helpful, thus let's remove the log.
Reviewed By: rkjfb
Differential Revision: D29317007
fbshipit-source-id: 3aba1ab8de7906e193946938aa69b32a09b8e5de
Summary:
Buck v2 builds from the root of the repo, not the current cell. This means that
the inferred logger name ends up being different.
We're going to need to fix this generally because otherwise it'll change logger
names for everyone (I'm tracking this in T93776519), but in the interest of not
having one Eden test arbitrarily failing on Buck v2 let's update this
with a workaround for now.
Reviewed By: genevievehelsel
Differential Revision: D29270388
fbshipit-source-id: 6968d9b6195a5eed7bd4018b161e12d88f78a421
Summary:
I'm suspecting that it is causing some broken pipe errors that aren't
resolving themself.
Reviewed By: kmancini
Differential Revision: D29279304
fbshipit-source-id: cfbf2261f2ac7dd7ec8b3311d1e27a0b9e160de4
Summary:
Invalidation on Windows is tricky, and I got it wrong in subtle ways
previously. The main obvious issue is that when the on-disk invalidation fails,
the refcount shouldn't be decremented as the placeholder/file is still present
on disk. This could cause weird issues in later checkout. The second one is how
invalidating a directory doesn't remove a placeholder (it actually adds one),
and thus we shouldn't decrement the FS refcount. And lastly, the refcount
should be decremented regardless of whether the inode is loaded or unloaded. As
long as it is known by the InodeMap it needs invalidation.
Reviewed By: fanzeyi
Differential Revision: D28970899
fbshipit-source-id: 0d64cadae01fcd4e028c53de9357ece7d648cdd4
Summary:
A user may have some undesirable environment variables when calling `edenfsctl start`,
which we do not want to propagate to edenfs as this may affect EdenFS's
ability to run properly. Having PYTHONPATH set to inside a repository may for
instance lead to a deadlock when EdenFS is trying to setup redirections.
To avoid this, we need to sanitize the environment before calling edenfs. This
functionality already exist but was bypassed on Windows.
Reviewed By: chadaustin
Differential Revision: D29244358
fbshipit-source-id: bc96698732e71412296ed5e28842b59b2c758699
Summary: The original commit broke globbing more than it fixed it. D29175333 will fully fix it, but in the meantime, let's revert the change to get a release out.
Reviewed By: singhsrb
Differential Revision: D29231954
fbshipit-source-id: 7a42e980c6fc4de09bee713a3a4141d52272b6d1
Summary:
Now that the InodeMap is pre-populated with the Overlay, we no longer need the
`folly::kIsWindows` test during checkout as inodes will enter this condition
when unloaded (IsInodeRemembered).
Reviewed By: fanzeyi
Differential Revision: D25106153
fbshipit-source-id: 511d795ae947651e1eaf3c54b8f1ab83c77f5cc4
Summary:
On a non-EdenFS mount, a case change of a file or directory isn't reported as
the file can still be accessed with the same path, and it has the same content
as before. EdenFS had a different behavior whereas renaming a file to a
different case would report a missing file and an untracked one, with different
casing.
This can be surprising to users as the behavior is different from Mercurial,
and it's also hard to actually fix as 2 renames would need to happen, a single
one would not work due to the case insensitivity nature of the filesystem.
While I do believe that reporting the case change might be more desirable to
allow users to actually commit changes the case of files, Mercurial might be
broken in subtle ways today if we allow this, thus it's best to make EdenFS
behave similarly to Mercurial.
Reviewed By: genevievehelsel
Differential Revision: D29150552
fbshipit-source-id: 6cceaa4c9fa61c03f35fcd91a9c01554da252222
Summary:
On case insensitive mounts, updating between commits that have a different
casing for a file/directory would lead to the update failing due to EdenFS
believing that an untracked file is present in the mount. That conflict is
however bogus and EdenFS simply gets confused in
TreeInode::processCheckoutEntry about the entry with the different casing.
To fix this, we should avoid comparing paths in a case sensitive manner and
instead compare then in a case insensitive fashion. This allows the rest of the
checkout code to update the directory/file in place.
On Windows, there is one more subtlety: we can't change the casing of a
file/directory that is already a placeholder and thus we need to force the
entire hierarchy to be removed, this will also make the checkout fail in case
of untracked files in the hierarchy, which is also the behavior on case
sensitive systems.
Reviewed By: genevievehelsel
Differential Revision: D29121741
fbshipit-source-id: 3d2cdacf296a3d061fc828cd6d04d249542cb63f
Summary:
## This diff
* We are migrating `edenfsctl strace` to cpp due to segfaulting from Thrift-py3 (similar to {D25515377 (a152fa4585)}).
* This diff implements new subcmd `edenfsctl trace strace` for this cpp migration.
Reviewed By: xavierd
Differential Revision: D29164534
fbshipit-source-id: 4d8ed23393004f394159c36f71e0c78c077c7c73
Summary:
When running tests, we need a way for Eden to find its privhelper. Normally, it
finds it by looking next to the EdenFS binary itself, but while that works in
Buck 1, it does not work in Buck 2, where the binaries are in separate
directories
Compared to the previous diffs, this requires some extra care because on a real
system, EdenFS is running as a setuid binary, and we don't want to let people
just run whatever binary they want the EdenFS privhelper!
To that end, we reject this argument entirely if the binary is setuid.
Reviewed By: xavierd
Differential Revision: D29061439
fbshipit-source-id: bf9427211d4209cf0bea805b9ea3a53270ec455b
Summary:
In Buck v1 the path we receive for hgPath is absolute so we can use it across
chdirs and the likes.
However, in Buck v2, it isn't. This means that using `--hgPath` doesn't work if
we go and chdir later, which we do because we run from the repo directory.
This updates the unit tests to set the path to an absolute value when they
otherwise look for their hg binary (which they do for operations in HgRepo). If
possible I would have preferred to thread this through to the HgImporter
constructor, but we have quite a few overloaded constructors already and this
turned out to be very messy.
Finally, this also updates the tests themselves to set `hgPath` to `false` so
that if we try to actually run it without configuring it, it won't work.
Reviewed By: xavierd
Differential Revision: D29060828
fbshipit-source-id: 17ad615d7ff98e9ce8278386f018167ec589c336
Summary:
I'd like to be able to override this from unit tests. The underlying goal is
to let us make this path absolute in unit tests, so that we can make Eden unit
tests that rely on hgPath work on Buck v2, where paths are relative.
xavierd suggested overriding the glfag would be the easiest course of action
so that's what this does.
Reviewed By: xavierd
Differential Revision: D29060827
fbshipit-source-id: c4084072e3eef9cc5c9d404d10b3e19028f177ea
Summary:
When matching against the ** pattern, EdenFS would match against the full path
instead of against just the current entry. This means that what was matched by
the glob prior to ** will be rechecked against by the pattern that follows **,
which isn't what is expected.
As a bonus, the `candidateName` variable will be constructed less than before
leading to less memory allocations.
Reviewed By: chadaustin
Differential Revision: D29079762
fbshipit-source-id: af15ecf229ce7119100dd375df23269bb7cdb1c0
Summary:
When using `eden glob` at the root of the repository, the `searchRoot` is
initialized with ".", which is an invalid PathComponent. Let's special case
this in edenfs itself as python's Path appear to always want to be "." and thus
it's easier to fix in EdenFS.
Reviewed By: chadaustin
Differential Revision: D29072478
fbshipit-source-id: 0108294e407784832cd35b3dd6aaceacf4bd7dd9
Summary:
These tests rely on the wall clock and fail somewhat regularly on CI
and on my local machine, so remove them.
Reviewed By: fanzeyi
Differential Revision: D29047847
fbshipit-source-id: 4008d079fac55f52827be534ab1b18b93c1d2f2f
Summary:
Whenever a file is modified in an EdenFS mount and a watchman subscription is
active, watchman will be nodified and will issue a getFilesChangedSince Thrift
call. In order to do that, Watchman ends up always re-creating a new connection
to EdenFS, causing the .eden/socket or .eden/config to be re-read in order to
find EdenFS's socket.
For workloads with heavy write traffic to EdenFS, this readlink/read can add
up. On Windows, writing ~2.5GB worth of data lead Watchman to read over 650MB
worth of data from the .eden/config!
Reviewed By: chadaustin
Differential Revision: D28912258
fbshipit-source-id: 549d57592672ef9cbdcccc213d0612d019677923
Summary:
## This diff
* Implement `getCauseDetail` for Fuse fetch context so that we log which Fuse call triggers the fetch.
Reviewed By: chadaustin, kmancini
Differential Revision: D29092690
fbshipit-source-id: 230ef193332a90a01ca9b89f5b8d8f20cdf4f321
Summary: The new name is `LockedPtr::as_lock`.
Reviewed By: aary
Differential Revision: D28987868
fbshipit-source-id: 8abd6a69a1b9c884adf137f06c24fe0df9ddd089
Summary: Buck has not relied on the .buckversion file for a while now. Trying to clean up the number of configs at the root of the cell for buck. This diff attempts to remove .buckversion code referecnes. Instead of calling `cat .buckversion` to get the buckversion hash, you can call `buck --fast-version` which parses the `buck-java11` file without downloading buck. Alternatively, you can also do something like ` cat .buck-java11 | grep -o -E -e "[0-9a-f]{40}" | head -1` to get the buckversion hash.
Reviewed By: DrMarcII
Differential Revision: D28553844
fbshipit-source-id: ab90fa6a5e7467f6d5c11ffa9d0e10a03433975f
Summary:
Using a PathMap to hold the list of mounts gives us case sensitivity for free
without needing to have #ifdef _WIN32, we can also avoid copying the paths just
to normalize them.
Reviewed By: fanzeyi
Differential Revision: D28969795
fbshipit-source-id: 859cdc3881e36db120913099ce3f451940fa48a4
Summary:
We're planning on rolling out NFS to our users soon, let's make sure NFS is
enabled.
Reviewed By: singhsrb
Differential Revision: D29002395
fbshipit-source-id: 8351fa45c9bd5e68e162baa80e20bd34564b5ece
Summary:
The InodeMap is fully functional on Windows, there is no need to ifdef out its
counters.
Reviewed By: genevievehelsel
Differential Revision: D28970898
fbshipit-source-id: adeb2db19c70d5ff4a3a162fe34c32195fc2a1e0
Summary:
On Windows, the working copy doesn't go away on unmount, instead placeholders
and full files[0] are still present on disk. For this reason, EdenFS needs to
either overly invalidate files and directories at update times, or need to
remember what is present on disk so the state can be recovered.
For now, this diff simply focus on reloading all the inodes that are present on
disk in the InodeMap.
[0]: https://docs.microsoft.com/en-us/windows/win32/projfs/cache-state
Reviewed By: chadaustin
Differential Revision: D28889082
fbshipit-source-id: 90170c1291da563bea455c8032dc8282a093c9b3
Summary: This will implement getTree API of RE-CAS
Reviewed By: chadaustin
Differential Revision: D28157535
fbshipit-source-id: 6b09338f0635aa948497d934a8351a9ba362f1ab
Summary: As we need to debugging a IO related performance issue. We want to see if writes in overlay has caused any issues. This diff adds an option to allow us to create in-memory overlays.
Reviewed By: chadaustin
Differential Revision: D28951905
fbshipit-source-id: 9ef1da183771ee69b855b49bbabc2ef02a7ec9ac
Summary:
Like it says in the title. Right now this dependency is invisible to Buck so
we rely on reading it from the checkout. However, while this works on Buck 1
(I guess we run the tests from the repo root), it doesn't work Buck 2, and it
won't work if we try to run those tests remotely.
Let's make the dependency explicit!
Reviewed By: xavierd
Differential Revision: D28936482
fbshipit-source-id: 71c4a492c33f586abe6b721b90898f5424c06e94
Summary:
Backing stores differentiate between individual tree objects and the
root of a checkout. For example, Git and Mercurial roots are commit
hashes. Allow EdenFS to track variable-width roots to better support
arbitrary backing stores.
Reviewed By: genevievehelsel
Differential Revision: D28619584
fbshipit-source-id: d94f1ecd21a0c416c1b4933341c70deabf386496
Summary:
Making it the default should make cloning using EdenFS easier and will remove
the need to manually pass the `--nfs` command line to `eden clone`.
Reviewed By: kmancini
Differential Revision: D28913818
fbshipit-source-id: 742c35c950cb5edf34fd9769fb78bf26095af0d5
Summary:
WAL is known to be significantly faster than the default DELETE journaling
mode, let's enable it.
Reviewed By: fanzeyi
Differential Revision: D28915718
fbshipit-source-id: 3ee10d10dbaf07a01a23101a6266874a41784cfc
Summary:
These appear to be unecessary, thus there is no need to keep them in place as
it makes the code harder to follow.
Reviewed By: chadaustin
Differential Revision: D28850241
fbshipit-source-id: 129b853e88f0291bb637c9d24de4872a222a5af4
Summary:
There are a few remaining holes where we are not passing a full fetch context.
We will need a full fetch context to do all data fetch logging needed for the
intern project. Additionally, we generally should not be using these singletons
in our production code.
A few final write calls need to be threaded as well.
Reviewed By: xavierd
Differential Revision: D28844011
fbshipit-source-id: 214853eea7fa65263fe6415e1ae8b76ca21512d6
Summary:
There are a few remaining holes where we are not passing a full fetch context.
We will need a full fetch context to do all data fetch logging needed for the
intern project. Additionally, we generally should not be using these singletons
in our production code.
Some thrift methods used by watchman to get metadata implicitly are not threaded.
These can cause fetches, so let's thread the fetch context here too.
Reviewed By: genevievehelsel
Differential Revision: D28842300
fbshipit-source-id: b1e4b3aea879d6ed7b92afa26184616dedad5935
Summary:
There are a few remaining holes where we are not passing a full fetch context.
We will need a full fetch context to do all data fetch logging needed for the
intern project. Additionally, we generally should not be using these singletons
in our production code.
This change is for symlink
Reviewed By: genevievehelsel
Differential Revision: D28841453
fbshipit-source-id: 080eb62f0b562f8e0995c34e9a8302238fc59ed8
Summary:
There are a few remaining holes where we are not passing a full fetch context.
We will need a full fetch context to do all data fetch logging needed for the
intern project. Additionally we generally should not be using these singletons
in our production code.
Most of rmdir is already threaded, not sure if this case can actuall cause
fetches in production, but might as well thread.
Reviewed By: genevievehelsel
Differential Revision: D28840211
fbshipit-source-id: 8dea08e775be470dd1730e2d32750a6912650ee0
Summary:
There are a few remaining holes where we are not passing a full fetch context.
We will need a full fetch context to do all data fetch logging needed for the
intern project. Additionally, we generally should not be using these singletons
in our production code.
this change is for rename
Reviewed By: genevievehelsel
Differential Revision: D23467437
fbshipit-source-id: e9d79c65fb5c4d686f0597550e43a0e87c4792cb
Summary:
we now want macFUSE to be installed instead of osxfuse. eden doctor will spew
errors if osxfuse is not installed or loaded, so it perpetually complains.
Update these checks for macFUSE now.
Reviewed By: xavierd
Differential Revision: D28751766
fbshipit-source-id: 4bc61349e33492aebe888a4e869ef7620c74768e
Summary:
There are a few remaining holes where we are not passing a full fetch context.
We will need a full fetch context to do all data fetch logging needed for the
intern project. Additionally, we generally should not be using these singletons
in our production code.
This change is for mkdir
Reviewed By: genevievehelsel
Differential Revision: D23458622
fbshipit-source-id: f3914a4f692490434882143664a5d5f1701e93ba
Summary:
There are a few remaining holes where we are not passing a full fetch context.
We will need a full fetch context to do all data fetch logging needed for the
intern project. Additionally we generally should not be using these singletons
in our production code.
This change is for create
Reviewed By: genevievehelsel
Differential Revision: D23457862
fbshipit-source-id: d4c9cc658c26b3119b2b2a1da061e299eaf510c9
Summary:
There are a few remaining holes where we are not passing a full fetch context.
We will need a full fetch context to do all data fetch logging needed for the
intern project. Additionally we generally should not be using these singletons
in our production code.
Most of lookup is already threaded. This finishes the threading for lookup.
Reviewed By: xavierd
Differential Revision: D23456910
fbshipit-source-id: fab7397caeee19f921d8fba1fb6528baa5cf2960
Summary:
BackingStore and LocalStore are no longer tied at the hip, so decouple
FakeBackingStore from LocalStore.
Reviewed By: kmancini
Differential Revision: D28615431
fbshipit-source-id: ee6bc807da6de4ed8fba8ab6d52ff5aeff34e8ae
Summary:
The meaning of the root ID is defined by the BackingStore, so move
parsing and rendering into the BackingStore interface.
Reviewed By: xavierd
Differential Revision: D28560426
fbshipit-source-id: 7cfed4870d48016811b604348742754f6cdbd842
