Summary:
This is much better than having `ObjectFetchContext` itself owns a copy of `ImportPriority`. We can actually customize how different fetch context manages these priority.
We set all FUSE requests to a higher priority and prefetch requests to a lower priority
Reviewed By: xavierd
Differential Revision: D22342802
fbshipit-source-id: b9c1d0f2ddbc7a5e5d619bc2c2222e5df0e702af
Summary:
This commit makes prefetch requests derived in `TreeInode` assigned to lower priority.
Since prefetch is not running under the original request's context, we create a new `ObjectFetchContext` specifically for this case, and attach it to `prefetchLease` since it is guaranteed to be alive during the span of prefetch.
Reviewed By: chadaustin
Differential Revision: D21872719
fbshipit-source-id: d1495b06031980d7d1c21ecf2a0b47e83fc9672d
Summary: This commit makes `readdir()` to correct send its `RequestContext` to `EdenDispatcher` method so underlying code can know the current request being processed is from FUSE.
Reviewed By: xavierd
Differential Revision: D21821949
fbshipit-source-id: f41ba912fedbfc040e3c9267aad25e7f33f8e912
Summary:
If we don't wait on these futures, they might never execute, which may lead to
EdenFS not being aware of files no longer present in the overlay.
For now, just add .get() to these futures, but really, we need to make these be
futures themselves and use continuation style code.
Reviewed By: fanzeyi
Differential Revision: D22326570
fbshipit-source-id: 7584981bec284f35f8d2689a3a5dafd1a111a3d8
Summary: The method name is symlink, not rmdir.
Reviewed By: genevievehelsel
Differential Revision: D22291297
fbshipit-source-id: 5dc37b053e06c965fd47df79990fc40adc097f87
Summary:
On Windows, paths are case insensitive (but the filesystem is case preserving),
and thus `open("FILE.TXT")` and `open("file.txt")` refer to the same file. When
that file is not materialized and its parent directory isn't yet enumerated,
PrjFS will call the PRJ_GET_PLACEHOLDER_INFO_CB with the file name passed in to
the `open` call. In this callback, if the passed in name refers to a valid
file, it needs to call PrjWritePlaceholderInfo to populate the directory entry.
Here is what the documentation for that function states:
"For example, if the PRJ_GET_PLACEHOLDER_INFO_CB callback specifies
dir1\dir1\FILE.TXT in callbackData->FilePathName, and the provider's backing
store contains a file called File.txt in the dir1\dir2 directory, and
PrjFileNameCompare returns 0 when comparing the names FILE.TXT and
File.txt, then the provider specifies dir1\dir2\File.txt as the value of
this parameter."
While the documentation doesn't state how that name is used internally, we can
infer (and test) that the returned case will be used as the canonical
representation of that file, ie: the one that a directory listing will see.
Since the PathMap code already does a case insensitive search, we just need to
make sure to use what it returns instead of re-using the name used for the search.
The only caveat to all of this is the original comment that describe that
`metadata.name` can't be used as it causes crashes. From what I can tell, this
was written in later 2018, and I believe is no longer relevant: the
`metadata.name` field was simply not populated.
Reviewed By: wez
Differential Revision: D21799627
fbshipit-source-id: aee877cc2d5f057944fcd39b1d59f0e97de6315c
Summary:
In following changes I will be threading ObjectFetchContext into the backing
store importing process, since this will start to be used more outside of the
ObjectStore, I am moving this class into its own files.
Reviewed By: chadaustin
Differential Revision: D22022488
fbshipit-source-id: 1a291fea6e0fd56855936962363dfc9f6de8533d
Summary:
All the unification of the mount code lead to this point, where supporting
unmount is merely just removing the various #ifdef.
Reviewed By: fanzeyi
Differential Revision: D21797078
fbshipit-source-id: 7602eb85e3fa276b72daebda36ae269ec38cde66
Summary:
On unmount, the code relies on the channel (fuse/prjfs) to set a promise, while
not strictly required on Windows as the interface is synchronous, this will
enable unmount to be the same between on all platforms.
Reviewed By: fanzeyi
Differential Revision: D21797081
fbshipit-source-id: c3eac448d2b13ef49db7a7d1f36bb084130ea63a
Summary:
Most of what fuseMount does can be shared between fuse and prjfs, so let's
rename it and start using it on Windows.
Reviewed By: fanzeyi
Differential Revision: D21797079
fbshipit-source-id: b92a36ecc702ee72df8ebdf9534add42bea9b97e
Summary:
This diff makes `lookup` to use `RequestData` as `ObjectFetchContext` in `getattr` calls.
This will make sure we correctly record backing store fetches in `eden top`
Reviewed By: chadaustin
Differential Revision: D21780969
fbshipit-source-id: 468e2fadcebf4a00477bc5de434e6c658b99d1ce
Summary: This diff makes `InodeBase::stat()` to be able to accept `ObjectFetchContext` as a parameter.
Reviewed By: chadaustin
Differential Revision: D21780883
fbshipit-source-id: 9b1db2e2268cb98663bcf902ea61897da593ea05
Summary: This diff starts to use `RequestData` as `ObjectFetchContext` in our `read()` methods. This ensures EdenFS could track backing store fetches happened in FUSE requests.
Reviewed By: chadaustin
Differential Revision: D21792503
fbshipit-source-id: 9509a1bc8f28100a0dfe196e312c4785c7842345
Summary: This diff passes a callback function `progressCallback` from `EdenServer` to make `OverlayChekcer` print fsck messages to stdout.
Reviewed By: fanzeyi
Differential Revision: D21966060
fbshipit-source-id: 317b2c8954c718b51b5295c9f8d7698eb203906e
Summary: This diff fixes `'__isset' is deprecated` problem when building `//eden/fs/service:edenfs`
Reviewed By: simpkins
Differential Revision: D21966922
fbshipit-source-id: c3b12134f3f84db0a97f3975dae5d64a2c98641a
Summary:
The mount path is required in 2 places: at mount time, and when deleting files.
It turns out that the second one doesn't require it as PrjDeleteFile specify
the filename to be: "A null-terminated Unicode string specifying the path,
relative to the virtualization root, to the file or directory to be deleted."
Thus, we only need it during mount time, so let's pass it to the start method.
Reviewed By: fanzeyi
Differential Revision: D21797080
fbshipit-source-id: 27f5d18841ac077c81ad3f6b3686e20917505c41
Summary:
This moves the edenToWinPath inside Windows specific files and makes
the interface clear about the kind of path that is expected.
Reviewed By: fanzeyi
Differential Revision: D21676318
fbshipit-source-id: d153bb543da5f481e94f83d52e45c6cc871d1f54
Summary:
The only reason tests need to set an FsChannel is for calling
cleanupPrjfsCache. For consistency sake, let's do the same as what we do on
unices: verify if we have a valid channel first.
Reviewed By: wez
Differential Revision: D21676316
fbshipit-source-id: 022ca91d32a7ab047a79acccd7d17dfd7906d527
Summary:
We want to use these on Windows too, so let's rename them to not be FUSE
specific.
Reviewed By: wez
Differential Revision: D21676317
fbshipit-source-id: 9f1bde9e0467d8828d4168a69581c8fea54669cd
Summary:
This moves it out of EdenServer and into EdenMount directly. This is closer
to what is happening in the Linux code and thus better.
Reviewed By: fanzeyi
Differential Revision: D21663501
fbshipit-source-id: 84420fcb37586cc2695568f0167b39f91c9bfb63
Summary:
Instead of having 2 different functions for Windows and other unices, let's
only have one. This allows more of the mount code to be shared between both.
Reviewed By: wez
Differential Revision: D21517477
fbshipit-source-id: 185ce864aecdf9a7880a9c705df444ec1e39319a
Summary:
Thrift setter API is deprecated since it doesn't bring any value over direct assignment. Removing it can reduce build-time and make our codebase more consistent.
If result of `s.set_foo(bar)` is unused, this diff replaces
s.set_foo(bar);
with
s.foo_ref() = bar;
Otherwise, it replaces
s.set_foo(bar)
with
s.foo_ref().emplace(bar)
Reviewed By: chadaustin
Differential Revision: D21712029
fbshipit-source-id: 3a332b4bf6fac6b3cf396d34e6d5ca4849181a6d
Summary: Fix some RVO warnings and a compilation error when compiling with gcc.
Reviewed By: xavierd
Differential Revision: D21481738
fbshipit-source-id: 0621f5886df40c24ef1a6a68ccd957e38f2f4122
Summary:
Next step in unifying the mount path, let's make the initialize the same in
Windows and unices. The only difference is now limited to the .eden directory
which we will be able to implement once regular users can create symlinks.
For the takeover code, the #ifdef is pushed down to the actual code that does
it, this allows the rest of the code to not have to bother about Windows vs
other platforms.
Reviewed By: wez
Differential Revision: D21517478
fbshipit-source-id: d40ca2694d23031ff98e319071e610efa306008f
Summary:
While this isn't the right fix, this is what shipped in our packages, for the
sake of being able to reproduce the package, let's land this as it is. A
future change will remove this ifdef.
Below is pkaush original description:
In Eden Windows we treat all the files as regular files and don't have a
concept of symlinks and executable files. Fixing the TreeEntryType::getType()
to return REGULAR_FILE for executable file and symlink.
Reviewed By: wez
Differential Revision: D20481051
fbshipit-source-id: 0b0c4d7aea28134383ef45aeafc02930b420286b
Summary:
On Windows, we've had a couple of cases where the overlay is corrupted which
prevent remounting EdenFS. In this case, one entry of a directory was an empty
path. Let's try to catch the error when it happens, instead of after the fact.
Reviewed By: wez
Differential Revision: D21443002
fbshipit-source-id: 47e59aee803cc0b4befd6e0585836d5c68e905b1
Summary:
All of these were simply NOT_IMPLEMENTED on Windows, but the code compiles
and doesn't break any existing tests. The underlying called functions might
have been implemented already, or are NOT_IMPLEMENTED, either way, this reduces
the amount of `#ifdef _WIN32`.
Reviewed By: chadaustin
Differential Revision: D21405622
fbshipit-source-id: bdc2de41d6a57e1c0b532e76eeb2c0c86180d558
Summary:
Only the mtime test had to be disabled due to getMetadata not being tracked
on Windows.
Reviewed By: wez
Differential Revision: D21312680
fbshipit-source-id: 5678956ed2b8b45b38d44c459557161aa6fd222e
Summary:
Fairly straightforward, one change to the test was necessary as reading an
unlinked file cannot be done on Windows.
Reviewed By: wez
Differential Revision: D21312681
fbshipit-source-id: c8cca5eeca7825983176ef618007c514b8e02140
Summary:
This brings it closer to folly::writeFile which should help in avoiding ifdef
whenever we want to use it.
Reviewed By: wez
Differential Revision: D21319020
fbshipit-source-id: 80fbf7fba671b18b5ef68375910e1a2a8869f590
Summary:
The MSVC compiler complains that it doesn't have the full definition of
PrivHelper, causing the build to fail. Include the right header to fix this.
Reviewed By: genevievehelsel
Differential Revision: D21381946
fbshipit-source-id: 0d0389ee8db44a36786973404c38487a94e8c4df
Summary:
While EdenFS does not use a separate privhelper process on Windows, it still
defines a stub PrivHelper class. However, this class was previously defined
in a separate win/utils/Stub.h header file, which led to awkward `#ifdef`s to
include the correct platform-specific header file.
This diff moves the definition of the dummy PrivHelper class in Windows into
the same `PrivHelper.h` header file used on POSIX platforms. This results in
a few more `ifdef`s in the PrivHelper files, but fewer `ifdef`s in the calling
code, and will make it easier to start unifying more of the `EdenMain` logic
on Windows and non-Windows platforms.
Reviewed By: xavierd
Differential Revision: D21332568
fbshipit-source-id: c63bf2b4a8b7e767d7db7dcda28675f735c23bf8
Summary:
FUSE_HANDLE_KILLPRIV expects that any write() call is handled by
clearing the setuid and setgid bits in the userspace. To avoid
implementing that behavior, disallow setting setuid or setgid in the
first place.
Reviewed By: xavierd
Differential Revision: D21333703
fbshipit-source-id: eb084ee8b00afe74c0da26e41c32c2cb742723da
Summary:
Move the `UserInfo` code from `fuse/privhelper` to `utils`, and also unify the
POSIX and Windows implementations of this class.
This code was originally put in the `fuse/privhelper` directory since it was
written at the same time as the privhelper. However, it's really a
lower-level library that doesn't depend on FUSE or any of the other code in
the `fuse/` subdirectory.
Reviewed By: wez
Differential Revision: D21296594
fbshipit-source-id: f58682f6ce86bba0328472c491bb4c0dc3370319
Summary:
The operation originally wanted to operate on the fuse `Attr`
structure which we don't have on Windows, so I repurposed the
`InodeBase::getattr` into `InodeBase::stat` and moved the conversion
of `struct stat` to `Dispatcher::Attr` to the `EdenDispatcher::getattr`
method (and a couple of other adhoc places that were doing a similar
conversion).
Reviewed By: chadaustin
Differential Revision: D20562459
fbshipit-source-id: 6b538110038352e9b5590fcb5ff5c33fe84ac1d8
Summary:
The only change I had to make was due to the fact that MSVC wasn't smart
enough to realize that the shift value couldn't be negative, so a manual
folly::to_unsigned was added to silence the warnings.
Reviewed By: simpkins
Differential Revision: D21268634
fbshipit-source-id: e65f15d58d5ea23bfa6796bab23cf1f5c2e7c12c
Summary:
On Windows, the expected refcount is one less than what it is on Linux, due
to the .eden directory not being present.
Reviewed By: simpkins
Differential Revision: D21268203
fbshipit-source-id: 91cfe742fa4d576917d552964d9541dc68ad2c75
Summary:
In the past, if an EdenMount got stuck while shutting down (inodes still referenced), the EdenMount would already be removed from the server's `mounPoint's` map while stuck. This resulted in `eden list` showing that the mount was not mounted, and then if we tried to remount the mount, we'd get a confusing error message about the not being able to grab the overlay lock. This also made it so if we tried to attach with gdb, or if we core dumped and tried to look at the data, we wouldn't be able to examine the object.
Instead, we should only remove the mount from the `mounPoints_` map after it has completely been shutdown. This will allow us to get better signal on the state of the mount (it will be in the `SHUTTING_DOWN` state in `eden list` instead of `(not mounted)`) and it will allow us to better debug these situations (where a mount is stuck shutting down) since we can get more information about the mount that is in this state.
Reviewed By: simpkins
Differential Revision: D21211544
fbshipit-source-id: 97ff11df67d0a80f47cf6b9f563876e17bd76622
Summary:
From looking at the code, it appears that the FUSE refcount is always a
uint32_t, except when serialized. Let's do the signedness conversion and
narrowing then.
Reviewed By: simpkins
Differential Revision: D21240161
fbshipit-source-id: 877c6cb6881cb36346c64cf92d99b1e588aed580
Summary:
The disabled test uses chmod and symlinks that aren't present in the WIN32
build. I'm not exactly sure why the .eden couldn't be found though.
Reviewed By: simpkins
Differential Revision: D21264070
fbshipit-source-id: da7385d9de4002c99fcbe79f37bd09acd41fbec8
Summary:
At the end of the test, the test assumes that an old uncommited version of
the file can be retrieved thanks to the files living in the overlay. On
Windows, the files are written directly onto the Projected FS cache, and thus
the old version of the file cannot be retrieved.
Reviewed By: pkaush
Differential Revision: D21193209
fbshipit-source-id: a4db6fb2bfc68817601725f97a6d47f2319b83c7
Summary:
While I had to disable a bunch of them at first, this allow them to compile
properly and run. Future diffs will attempt to enable the disabled ones.
Reviewed By: pkaush
Differential Revision: D21188118
fbshipit-source-id: 154fec49c76563b0856fa36e78b2bbd09f0f2381
Summary:
This enables globFiles for Windows, with some
minor tweaks around dtype to enable the build and make
the results consistent between watchman and eden.
Reviewed By: chadaustin
Differential Revision: D20536715
fbshipit-source-id: b1c8184dc664910e4d052a21b4cd993ddfaadf25
Summary: This diff ports DiffTest on Windows and enables them for the CMake builds.
Reviewed By: simpkins
Differential Revision: D20480869
fbshipit-source-id: 77eda5ef53e4cd19552f0da52de4ff53d9807cc3
Summary: This diff adds readdir tests for Windows and enables InodeBaseTest and TreeInodeTest for CMake build.
Reviewed By: simpkins
Differential Revision: D20480866
fbshipit-source-id: 44fb33448c2e6cdb3ecce08136bf8b9e1d5c398a
