Summary:
Now that sqliteoverlay has a TARGETS file, we can remove the manual, and use
autodeps to have the right dependencies in the TARGETS files. This will help in
getting EdenFS to build on Windows with buck.
Reviewed By: chadaustin
Differential Revision: D23820312
fbshipit-source-id: 34bfd13d2ae6d11a404a9b913562c7d45a4b3de7
Summary:
This commit moves a compile-time template parameter
to be a runtime boolean parameter.
There's a bit of fan-out that, while I don't think it is
super awesome, isn't super terrible either.
The case sensitivity value is read from the checkout config
added in the prior diff in this stack.
Reviewed By: xavierd
Differential Revision: D23751192
fbshipit-source-id: 46f6fe25bfa6666305096ad9c416b510cd3aac8f
Summary:
This diff teaches the CheckoutConfig how to determine
whether a given checkout should be case-sensitive (the default)
or case-insensitive-case-preserving.
This option is passed through to the fuse channel initialization,
so that the kernel will respect it, however, our DirEntry layer
doesn't yet know that it should respect this.
There's currently no UI to set this option. My game plan
is to suggest the following steps to folks that want to try
this out:
```
$ eden stop
$ vim ~/local/.eden/clients/ovrsource/config.toml
```
and then add this line to the `[repository]` section:
```
case-sensitive = false
```
and finally:
```
$ eden start
```
Reviewed By: xavierd
Differential Revision: D23751184
fbshipit-source-id: 6facb23c460cfff6e37d0091b51b97ab06f62c91
Summary:
Now that all the pieces are in place, we can plumb the request context in. For
now, this adds it to only one callback as I figure out more about it and tweak
it until I have something satisfactory. There are some rough edges with it that
I'm not entirely happy about, but as I change the notification callback to be
more async, I'm hoping to make more convenient to use and less clanky.
Reviewed By: fanzeyi
Differential Revision: D23505508
fbshipit-source-id: d5f12e22a8f67dfa061b8ad82ea718582c323b45
Summary:
Since the Stub.h now only contains NOT_IMPLEMENTED, let's move it to its own
header outside of the win directory.
Reviewed By: genevievehelsel
Differential Revision: D23696244
fbshipit-source-id: 2dfc3204707e043ee6c89595668c484e0fa8c0d0
Summary:
With this gone, we will be able to rename and move Stub.h outside of the win
directory.
Reviewed By: genevievehelsel
Differential Revision: D23696243
fbshipit-source-id: ea05b10951fa38a77ce38cd6a09a293364dbeec9
Summary:
While the code isn't compiled, this makes the thrift definition available to
the rest of the code, eliminating the need for having a stub for
SerializedInodeMap on Windows.
Reviewed By: genevievehelsel
Differential Revision: D23696242
fbshipit-source-id: 8a42dd2ed16887f3b7d161511e07aaa35fd1b968
Summary:
This was only used in EdenMount.h, to declare a method that was not compiled on
Windows, let's ifdef that method instead.
Reviewed By: chadaustin
Differential Revision: D23693494
fbshipit-source-id: 1eda62f2ae3a38a30aa0b517911635ef3d3896c2
Summary: A following diff will make use of the channel.
Reviewed By: wez
Differential Revision: D23505510
fbshipit-source-id: c044fff51c8771b1ead86333317e5c617184075c
Summary:
Readdir tries to be smart and prefetch the metadata for each of the children.
But this uses the old path to read metadata for files which can cause eden to
download the blob. When metadata prefetching is turned on in the backing store
it is better to leave this to metadata prefetching to the backing store.
Reviewed By: wez
Differential Revision: D23476876
fbshipit-source-id: 41cc5e6f423f19adb18581564c069c12621b6c1b
Summary:
Most of the RequestData code is platform generic, but bits of it are currently
strongly tied to FUSE. By splitting these 2 parts, we will be able to use the
RequestContext class in Windows too and not having to re-implement all the
logic there.
Reviewed By: chadaustin
Differential Revision: D23482072
fbshipit-source-id: 857fd9ca4264d0f308ec10cc487e9ff3eeb5ee16
Summary:
We are unifying C++ APIs for accessing optional and unqualified fields:
https://fb.workplace.com/groups/1730279463893632/permalink/2541675446087359/.
This diff migrates code from accessing data members generated from unqualified
Thrift fields directly to the `field_ref` API, i.e. replacing
```
thrift_obj.field
```
with
```
*thrift_obj.field_ref()
```
The `_ref` suffixes will be removed in the future once data members are private
and names can be reclaimed.
The output of this codemod has been reviewed in D20039637.
The new API is documented in
https://our.intern.facebook.com/intern/wiki/Thrift/FieldAccess/.
drop-conflicts
Reviewed By: simpkins
Differential Revision: D23465292
fbshipit-source-id: bb9df3ad183685fae28173da7275e6ecd34df048
Summary:
None of these were used, let's remove them.
ps: I thought we had a system to detect unused headers and lint about them?
Reviewed By: chadaustin
Differential Revision: D23465783
fbshipit-source-id: c21a34c9838db29f4fd0057d3be4e0fcb527cd6d
Summary:
This is not per-se fuse related, thus move it to a common location and remove
the duplicated define in FileInode.h
Reviewed By: chadaustin
Differential Revision: D23465192
fbshipit-source-id: 5fa7709f127c2d3372ee5ea3aeb89e793ea5b9f7
Summary: This file is not fuse specific, therefore, let's move it to a non-fuse folder.
Reviewed By: chadaustin
Differential Revision: D23464460
fbshipit-source-id: f70e94bb0ecc37bd74798fd230dee2058484f31b
Summary:
This commit introduces a new process spawning class derived
from the ChildProcess class in the watchman codebase.
`SpawnedProcess` is similar to folly::Subprocess but is designed around the
idea that we will use a system provided spawning API to start a process, rather
than assuming the use of `fork`.
`fork` is to be avoided because it can be expensive for processes with large
address spaces and also because it interacts poorly with threads on macOS. In
particular, we see the objC runtime terminating our process in some scenarios
where fork and threads are mixed.
There are some important differences from `folly::Subprocess` and that means
that some assumptions and uses need to be altered slightly from their prior
workings. For example, detaching a SpawnedProcess moves the responsibility of
waiting on the child to a periodic task as there is no way to detach via
posix_spawn without also using fork.
On the plus side, this commit allows unifying spawning between posix and
windows systems, which simplifies the code!
Reviewed By: xavierd
Differential Revision: D23287763
fbshipit-source-id: b662af1d7eaaa9ed445c42f6c5765ae9af975eea
Summary:
As previous diffs in the stack show there were at least one place in the
codebase which used incorrect object context logger and that resulted in "blind
spots" in undesired file fetches logging i.e. undesired file fetches were
logged, but neither pid nor cmd-line was logged.
There are quite a few places in the codebase that use null
object fetch context, and threading the correct object fetch context to all of
them might be hard. Threading the context is a bit annoying, so it would be good to know something like "EdenDispatcher code is responsible for most of the blind spots, so let's thread the correct context there first". Or it would be equally good to know that none of the null object context are responsible for blind spots.
This diff might help us decide where we need to thread real object fetch context
first. Instead of passing null object fetch context let's pass null object
fetch context with causeDetail field. This field will be logged to scuba (see
BackingStoreLogger::logImport code), and instead of getting "Unknown" interface
we'll get e.g. "Unknown - EdenDispatcher::create", and that would highlight
where we need to thread the context.
A note about implementation - getNullContextWithCauseDetail returns a raw pointer
which is expected to be static i.e. it should work similarly to current
getNullContext implementation. It's quite a hack, but allows us to get rid of
memory allocations (we'd have one memory allocation per place in the code where
getNullContextWithCauseDetail). Let me know if you are ok with this hack.
Reviewed By: kmancini
Differential Revision: D23422526
fbshipit-source-id: e576bba9fc09e160fc42771c7589cdd1694d93c0
Summary:
Scuba logging that tracks undesired file fetches has some blind spots i.e. a
lot of fetches have null pid and null cmd line. This diff tries to fix another
part of the problem.
TreeInode::getOrLoadChild() has TODO `pass a fetch context down through
getOrLoadChild to track this load`. This diff fixes this TODO, and also starts
to pass context from EdenDispatcher:lookup method.
Note that it adds quite a lot of new `ObjectFetchContext::getNullContext()`
calls, and potentially those might be responsible for blind spots in logging.
I'll try to address this problem in the next diffs.
Reviewed By: kmancini
Differential Revision: D23418218
fbshipit-source-id: 319d7436494d8dce3580289aae9963aa13bfc191
Summary:
Scuba logging that tracks undesired file fetches has some blind spots i.e. a
lot of fetches have null pid and null cmd line. This diff fixes at least part
of the problem.
TreePrefetchContext which is used from TreePrefetchLease didn't logged client
pid at all (in fact, it logged almost nothing). This diff fixes at least one blind spot, however it doesn't look like this is the only one.
Reviewed By: kmancini
Differential Revision: D23417451
fbshipit-source-id: 107884e94c6b40de999328ec2ef78fe22174c1ca
Summary:
Setting up, tearing down, and querying RequestContext has some
overhead that I would like to avoid in the inner FUSE loop, so replace
RequestData with a single class that's heap-allocated at the start of
a request and is guaranteed to survive until the request ends, and is
otherwise explicitly passed where it's needed.
Reviewed By: kmancini
Differential Revision: D22712310
fbshipit-source-id: fc30d0b0f7e22b39306b857194ea07a913110b0f
Summary: Use `Subprocess` in `win/utils` to call `edenfsctl redirection fixup` after mount is done.
Reviewed By: wez
Differential Revision: D22958764
fbshipit-source-id: a485994a3816169299e8514a5c355f3d37edad99
Summary:
Up to now, Windows had to have its own version of folly::{readFile, writeFile,
writeFileAtomic} as these only operate on `char *` path, which can only
represent ascii paths on Windows. Since the Windows version is slightly
different from folly, this forced the code to either ifdef _WIN32, or use the
folly version pretending that it would be OK. The Windows version was also
behaving slightly differently from folly. For instance, where folly would
return a boolean to indicate success, on Windows we would throw an exception.
To simplify our code, add type safety and unify both, we can implement our own
wrappers on top of either folly or Windows APIs.
We still have some code that uses folly::readFile but these should only be
dealing with filedescriptors. As a following step, we may want to have our own
File class that wraps a file descriptor/HANDLE so we can completely remove all
uses of folly::readFile.
Reviewed By: wez
Differential Revision: D23037325
fbshipit-source-id: 2b9a026f3ee6220ef55097abe649b23e38d9fe91
Summary:
After enabling strace profiling for the dispatcher, it became very clear that
once files are populated on disk, most of the notification in edenfs come from
listing directories. While looking at the code to find any improvements, I
picked up a couple of low hanging fruits and cleanup that should help (not
measured).
Reviewed By: chadaustin
Differential Revision: D23121434
fbshipit-source-id: 8a8b369f3be6ffb6097e6e12ab0f07af0a6d56b7
Summary:
From a quick experiment, this greatly cuts down on the amount requests to
nonexistent files. For instance, the .hg directory in folders is now only
looked up once and no longer afterwards.
Reviewed By: wez
Differential Revision: D23112343
fbshipit-source-id: 223134ca591054ae9ac2e839033bbd1b714443da
Summary:
This is purely moving code around to be more similar between Unices/Windows.
At some point in the future, I'd like to see just one EdenDispatcher class in
the shared code with as few platform specific code (ie: ifdef) as possible,
this is small step in this direction.
Reviewed By: wez
Differential Revision: D23112345
fbshipit-source-id: feb276040aea106d5951af30c5d445b4cb654e01
Summary:
On Linux, all the notifications are sent to the strace logger, let's do the
same on Windows. Whenever possible, I've tried to use the name of the Linux
syscalls that correspond to the notification.
Reviewed By: chadaustin
Differential Revision: D23105536
fbshipit-source-id: 31c6e545cdec93b0f751682e34c34b894b2890c4
Summary:
The StringConv.h header contains many functions to convert from Windows paths
to Eden's path (and vice versa) to workaround the fact that Eden's path don't
support wide strings that Windows uses. Let's simply add support for these wide
strings in PathFuncs so we can greatly simplify all the call sites. Instead of
calling "edenToWinName(winstr)", "PathComponent(winstr)" is both more
descriptive and more idiomatic.
To be fair, I'm not entirely a fan of the approach taken in this diff, as this
adds Windows specific code to PathFuncs.h, but I feel that the benefit is too
big to not do that.
Reviewed By: chadaustin
Differential Revision: D23004523
fbshipit-source-id: 3a1507e398a66909773251907db01e06603b91dd
Summary:
Reading the entire file to compute its sha1 can cause EdenFS to OOM on large
files. Instead, let's simpy read 8k at a time, like what we do on Linux.
Reviewed By: chadaustin
Differential Revision: D22924710
fbshipit-source-id: cd472f396a4385049994a1976c7d046cb901337a
Summary:
The initialized_ field was never set, which meant that Overlay::close would
never close the sqlite overlay. This had the negative effect of allowing the
unmount thrift call to return before the sqliteoverlay is properly closed which
on Windows meant that the overlay couldn't be deleted.
Reviewed By: chadaustin
Differential Revision: D22915353
fbshipit-source-id: e501b2d0268449d109cc37dfdc62faebf1953e09
Summary:
Cache invalidation is hard, and on Windows we avoided doing a lot of them. It
turns out, this was the wrong decision as it's fairly easy to find cases where
the filesystem view is different from the manifest state.
Since the Linux code is most likely correct in where the invalidation is done,
let's also do the same on Windows, removing a whole lot of #ifdef. It is very
likely that as a result of this diff we end up invalidating more than needed,
thus slowing down EdenFS, but at this point I'd prefer to err on the side of
correctness, performance will come later.
While invalidating files should use PrjDeleteFile, for directories, we simply
need to mark them as placeholder, as directories created by a user won't have a
placeholder, thus ProjectedFS would bypass EdenFS when listing in.
Reviewed By: chadaustin
Differential Revision: D22833202
fbshipit-source-id: d807557f5e44279c49ab701b7a797253ef1f0717
Summary:
Avoid some overhead and complexity by storing BufVec as a
unique_ptr<IOBuf>. The complexity can be reintroduced if we ever find
FUSE splice support to be a performance win for us.
Reviewed By: kmancini
Differential Revision: D22710795
fbshipit-source-id: e58eedc0fb5cea9e9743ccd20d3e4e2b7cc5d198
Summary:
Avoid the cost of dynamically querying whether we are in a FUSE
request handler or not by passing a flag.
Reviewed By: kmancini
Differential Revision: D22710480
fbshipit-source-id: 010bb8efee8074441aa20aab0eb12277452c5252
Summary:
Avoid the cost of dynamically querying whether we are in a FUSE
request handler or not by passing a flag.
Reviewed By: kmancini
Differential Revision: D22710452
fbshipit-source-id: 818035b72b793fa895147d9df3bb668d5b9c55f3
Summary:
Avoid the cost of dynamically querying whether we are in a FUSE
request handler or not by passing a flag.
Reviewed By: kmancini
Differential Revision: D22710422
fbshipit-source-id: 65b0737ad5f8ca74d12f2c657691d3751df4aa54
Summary:
Avoid the cost of dynamically querying whether we are in a FUSE
request handler or not by passing a flag.
Reviewed By: genevievehelsel
Differential Revision: D22710397
fbshipit-source-id: 7c62f45dfc227416c91070842a349b9d0c626cba
Summary: This is unused, and won't ever be used.
Reviewed By: genevievehelsel
Differential Revision: D22744352
fbshipit-source-id: 9d20db608f972288eaf33e3ea0a79ffe5e13e03e
Summary:
Both unices and Windows needs to invalidate the cache in the same place, let's
avoid ifdef and consolidate the function name to clean things up a bit.
Reviewed By: chadaustin
Differential Revision: D22741709
fbshipit-source-id: 04060c0080eff9840abd22747ea48404fa50fd86
Summary:
We're still reading the entire file at a time, but this paves the way to not do
that.
While the change looks big, a lot of it is just moving code around. The main
gist of it is removing EdenMount::readFile and writing the proper future
combinator in EdenDispatcher::getFileData.
Reviewed By: wez
Differential Revision: D22361748
fbshipit-source-id: 6391a29d25a4c9e61b91952c40c21ad52e728c8b
Summary:
This would enable the enumeration callback to use ProjectedFS asynchronous
completion.
Reviewed By: chadaustin
Differential Revision: D22361747
fbshipit-source-id: b2d31533ee5128e9dd3da7f91d5225331cf8e926
Summary:
We've had a long latent bug where for some reason a file that should be in the
repo isn't visible on the filesystem. Turning on logging and running `ls` shows
that EdenFS tells ProjectedFS about the file, but it's somehow not shown. The
reason being that ProjectedFS keeps track of all the removed files by adding a
tombstone for these, and will not list them until EdenFS removes this
tombstone.
Looking at the various places where we invalidate FUSE's cache, but not
ProjectedFS, only one stands out, and it has to do with files not being present
in the working copy, but added in the update destination. Adding the right
flush there appear to solve the simple repro listed below.
I'll remove all the ifdef around flushing in a later diff to avoid this issue
from re-appearing again.
Reviewed By: fanzeyi
Differential Revision: D22739916
fbshipit-source-id: 3a4fbc825cd21b36cbd2616882fd50e3d9741f63
Summary:
This adds logging for data fetches that come from the thrift checkout call to
help debug the cause of unexpected data fetches. (See D22448048 for more
motivation)
Reviewed By: chadaustin
Differential Revision: D22489504
fbshipit-source-id: 3b732a1e5627c2130f561ec0138a1df270e1925d
Summary: This is unused, no need to keep it around.
Reviewed By: wez
Differential Revision: D22361749
fbshipit-source-id: 3c353776437b59c6c7735652f7eb1ce052215e11
Summary:
Futures are intended to be chained together and not synchronously waited one
after the other. While this may be one of the goal, it also paves the way to
enable ProjectedFS asynchronous notification handling.
While doing this, a bunch of code was moved from EdenMount.cpp to the
dispatcher itself, the rationale behind this is to follow what the unix
EdenDispacher with the long term plan to merge the 2 as much as possible.
Reviewed By: wez
Differential Revision: D22361750
fbshipit-source-id: fa679a8b94ff6f8b5a33782fdb6b129ab066c4d8
Summary:
At mount time, EdenFS will try to create the .eden/config file which will
indicate to the client that this is an EdenFS repo, this config also contains
the location of the socket to talk to EdenFS.
On unices, and while the .eden directory is slightly different, the content of
this directory is written onto the overlay, but on Windows, the overlay is the
repository itself. What this means is that creating this config file will end
up triggering a ProjFS callback, which can potentially lead to deadlocks if
we're not careful.
A future change will simply prevent these recursive callbacks from happening
and thus a solution needs to be found for the .eden/config file itself. Since
the file itself is tiny (about 400B), and should only be accessed once[0], the
simple approach to keep it in memory and special case it works perfectly.
[0]: Once a file was read fully by ProjFS, it's present in the overlay and
EdenFS will no longer be requested to provide it.
Reviewed By: chadaustin
Differential Revision: D22310734
fbshipit-source-id: 6b2dba2164496ebd251104d7875b51569be2471f
Summary:
within() schedules work on the singleton timekeeper thread which
sometimes scheduled a timer callback on the ServerExecutor while
TestMount was shutting down, causing an assertion failure. Since
TestMount's executor is deterministic, just make the entire test
deterministic.
Reviewed By: genevievehelsel
Differential Revision: D22709150
fbshipit-source-id: 0a1621df0a9934bb7944fc259f7f83d110b35e93
Summary:
We are unifying C++ APIs for accessing optional and unqualified fields:
https://fb.workplace.com/groups/1730279463893632/permalink/2541675446087359/.
This diff migrates code from accessing data members generated from unqualified
Thrift fields directly to the `field_ref` API, i.e. replacing
```
thrift_obj.field
```
with
```
*thrift_obj.field_ref()
```
The `_ref` suffixes will be removed in the future once data members are private
and names can be reclaimed.
The output of this codemod has been reviewed in D20039637.
The new API is documented in
https://our.intern.facebook.com/intern/wiki/Thrift/FieldAccess/.
drop-conflicts
Reviewed By: yfeldblum
Differential Revision: D22631599
fbshipit-source-id: 9bfcaeb636f34a32fd871c7cd6a2db4a7ace30bf
Summary:
This function is only used in EdenDispatcher, and all of it can be achieved by
simply calling InodeBase::stat method to fetch the file size.
As a bonus, the code now uses future continuation, instead of individual calls
to `get`.
Reviewed By: chadaustin
Differential Revision: D22310733
fbshipit-source-id: c81be7dd114c3a9bebc6b1b5b0887ae3ade30602
Summary:
The WinStore was only used for 2 simple things: checking that a file is
present, and getting its blob. Since both of these are trivial to implement
with EdenMount, there is no reason for WinStore to exist.
Reviewed By: chadaustin
Differential Revision: D22288430
fbshipit-source-id: 90567ac39b5124039becd3599766fcd0cde3e9aa
Summary: This diff defines `Overlaychecker::ProgressCallback` to replace repetitive function type declaration.
Reviewed By: genevievehelsel
Differential Revision: D22243160
fbshipit-source-id: ea05e451817a760b5266879b956eaea48dc8d85e
Summary: On macOS, "Icon?" (aka "Icon\r") is a sometimes added. This file is weird to ignore, and should be ignored using "Icon\r\r" or "Icon[/r]". This won't be hidden with "Icon\r" or "Icon\r"
Reviewed By: chadaustin
Differential Revision: D22050682
fbshipit-source-id: 51d7d4c2414a07b959120455ae991d2425c1ea4d
Summary:
This is much better than having `ObjectFetchContext` itself owns a copy of `ImportPriority`. We can actually customize how different fetch context manages these priority.
We set all FUSE requests to a higher priority and prefetch requests to a lower priority
Reviewed By: xavierd
Differential Revision: D22342802
fbshipit-source-id: b9c1d0f2ddbc7a5e5d619bc2c2222e5df0e702af
Summary:
This commit makes prefetch requests derived in `TreeInode` assigned to lower priority.
Since prefetch is not running under the original request's context, we create a new `ObjectFetchContext` specifically for this case, and attach it to `prefetchLease` since it is guaranteed to be alive during the span of prefetch.
Reviewed By: chadaustin
Differential Revision: D21872719
fbshipit-source-id: d1495b06031980d7d1c21ecf2a0b47e83fc9672d
Summary: This commit makes `readdir()` to correct send its `RequestContext` to `EdenDispatcher` method so underlying code can know the current request being processed is from FUSE.
Reviewed By: xavierd
Differential Revision: D21821949
fbshipit-source-id: f41ba912fedbfc040e3c9267aad25e7f33f8e912
Summary:
If we don't wait on these futures, they might never execute, which may lead to
EdenFS not being aware of files no longer present in the overlay.
For now, just add .get() to these futures, but really, we need to make these be
futures themselves and use continuation style code.
Reviewed By: fanzeyi
Differential Revision: D22326570
fbshipit-source-id: 7584981bec284f35f8d2689a3a5dafd1a111a3d8
Summary: The method name is symlink, not rmdir.
Reviewed By: genevievehelsel
Differential Revision: D22291297
fbshipit-source-id: 5dc37b053e06c965fd47df79990fc40adc097f87
Summary:
On Windows, paths are case insensitive (but the filesystem is case preserving),
and thus `open("FILE.TXT")` and `open("file.txt")` refer to the same file. When
that file is not materialized and its parent directory isn't yet enumerated,
PrjFS will call the PRJ_GET_PLACEHOLDER_INFO_CB with the file name passed in to
the `open` call. In this callback, if the passed in name refers to a valid
file, it needs to call PrjWritePlaceholderInfo to populate the directory entry.
Here is what the documentation for that function states:
"For example, if the PRJ_GET_PLACEHOLDER_INFO_CB callback specifies
dir1\dir1\FILE.TXT in callbackData->FilePathName, and the provider's backing
store contains a file called File.txt in the dir1\dir2 directory, and
PrjFileNameCompare returns 0 when comparing the names FILE.TXT and
File.txt, then the provider specifies dir1\dir2\File.txt as the value of
this parameter."
While the documentation doesn't state how that name is used internally, we can
infer (and test) that the returned case will be used as the canonical
representation of that file, ie: the one that a directory listing will see.
Since the PathMap code already does a case insensitive search, we just need to
make sure to use what it returns instead of re-using the name used for the search.
The only caveat to all of this is the original comment that describe that
`metadata.name` can't be used as it causes crashes. From what I can tell, this
was written in later 2018, and I believe is no longer relevant: the
`metadata.name` field was simply not populated.
Reviewed By: wez
Differential Revision: D21799627
fbshipit-source-id: aee877cc2d5f057944fcd39b1d59f0e97de6315c
Summary:
In following changes I will be threading ObjectFetchContext into the backing
store importing process, since this will start to be used more outside of the
ObjectStore, I am moving this class into its own files.
Reviewed By: chadaustin
Differential Revision: D22022488
fbshipit-source-id: 1a291fea6e0fd56855936962363dfc9f6de8533d
Summary:
All the unification of the mount code lead to this point, where supporting
unmount is merely just removing the various #ifdef.
Reviewed By: fanzeyi
Differential Revision: D21797078
fbshipit-source-id: 7602eb85e3fa276b72daebda36ae269ec38cde66
Summary:
On unmount, the code relies on the channel (fuse/prjfs) to set a promise, while
not strictly required on Windows as the interface is synchronous, this will
enable unmount to be the same between on all platforms.
Reviewed By: fanzeyi
Differential Revision: D21797081
fbshipit-source-id: c3eac448d2b13ef49db7a7d1f36bb084130ea63a
Summary:
Most of what fuseMount does can be shared between fuse and prjfs, so let's
rename it and start using it on Windows.
Reviewed By: fanzeyi
Differential Revision: D21797079
fbshipit-source-id: b92a36ecc702ee72df8ebdf9534add42bea9b97e
Summary:
This diff makes `lookup` to use `RequestData` as `ObjectFetchContext` in `getattr` calls.
This will make sure we correctly record backing store fetches in `eden top`
Reviewed By: chadaustin
Differential Revision: D21780969
fbshipit-source-id: 468e2fadcebf4a00477bc5de434e6c658b99d1ce
Summary: This diff makes `InodeBase::stat()` to be able to accept `ObjectFetchContext` as a parameter.
Reviewed By: chadaustin
Differential Revision: D21780883
fbshipit-source-id: 9b1db2e2268cb98663bcf902ea61897da593ea05
Summary: This diff starts to use `RequestData` as `ObjectFetchContext` in our `read()` methods. This ensures EdenFS could track backing store fetches happened in FUSE requests.
Reviewed By: chadaustin
Differential Revision: D21792503
fbshipit-source-id: 9509a1bc8f28100a0dfe196e312c4785c7842345
Summary: This diff passes a callback function `progressCallback` from `EdenServer` to make `OverlayChekcer` print fsck messages to stdout.
Reviewed By: fanzeyi
Differential Revision: D21966060
fbshipit-source-id: 317b2c8954c718b51b5295c9f8d7698eb203906e
Summary: This diff fixes `'__isset' is deprecated` problem when building `//eden/fs/service:edenfs`
Reviewed By: simpkins
Differential Revision: D21966922
fbshipit-source-id: c3b12134f3f84db0a97f3975dae5d64a2c98641a
Summary:
The mount path is required in 2 places: at mount time, and when deleting files.
It turns out that the second one doesn't require it as PrjDeleteFile specify
the filename to be: "A null-terminated Unicode string specifying the path,
relative to the virtualization root, to the file or directory to be deleted."
Thus, we only need it during mount time, so let's pass it to the start method.
Reviewed By: fanzeyi
Differential Revision: D21797080
fbshipit-source-id: 27f5d18841ac077c81ad3f6b3686e20917505c41
Summary:
This moves the edenToWinPath inside Windows specific files and makes
the interface clear about the kind of path that is expected.
Reviewed By: fanzeyi
Differential Revision: D21676318
fbshipit-source-id: d153bb543da5f481e94f83d52e45c6cc871d1f54
Summary:
The only reason tests need to set an FsChannel is for calling
cleanupPrjfsCache. For consistency sake, let's do the same as what we do on
unices: verify if we have a valid channel first.
Reviewed By: wez
Differential Revision: D21676316
fbshipit-source-id: 022ca91d32a7ab047a79acccd7d17dfd7906d527
Summary:
We want to use these on Windows too, so let's rename them to not be FUSE
specific.
Reviewed By: wez
Differential Revision: D21676317
fbshipit-source-id: 9f1bde9e0467d8828d4168a69581c8fea54669cd
Summary:
This moves it out of EdenServer and into EdenMount directly. This is closer
to what is happening in the Linux code and thus better.
Reviewed By: fanzeyi
Differential Revision: D21663501
fbshipit-source-id: 84420fcb37586cc2695568f0167b39f91c9bfb63
Summary:
Instead of having 2 different functions for Windows and other unices, let's
only have one. This allows more of the mount code to be shared between both.
Reviewed By: wez
Differential Revision: D21517477
fbshipit-source-id: 185ce864aecdf9a7880a9c705df444ec1e39319a
Summary:
Thrift setter API is deprecated since it doesn't bring any value over direct assignment. Removing it can reduce build-time and make our codebase more consistent.
If result of `s.set_foo(bar)` is unused, this diff replaces
s.set_foo(bar);
with
s.foo_ref() = bar;
Otherwise, it replaces
s.set_foo(bar)
with
s.foo_ref().emplace(bar)
Reviewed By: chadaustin
Differential Revision: D21712029
fbshipit-source-id: 3a332b4bf6fac6b3cf396d34e6d5ca4849181a6d
Summary: Fix some RVO warnings and a compilation error when compiling with gcc.
Reviewed By: xavierd
Differential Revision: D21481738
fbshipit-source-id: 0621f5886df40c24ef1a6a68ccd957e38f2f4122
Summary:
Next step in unifying the mount path, let's make the initialize the same in
Windows and unices. The only difference is now limited to the .eden directory
which we will be able to implement once regular users can create symlinks.
For the takeover code, the #ifdef is pushed down to the actual code that does
it, this allows the rest of the code to not have to bother about Windows vs
other platforms.
Reviewed By: wez
Differential Revision: D21517478
fbshipit-source-id: d40ca2694d23031ff98e319071e610efa306008f
Summary:
While this isn't the right fix, this is what shipped in our packages, for the
sake of being able to reproduce the package, let's land this as it is. A
future change will remove this ifdef.
Below is pkaush original description:
In Eden Windows we treat all the files as regular files and don't have a
concept of symlinks and executable files. Fixing the TreeEntryType::getType()
to return REGULAR_FILE for executable file and symlink.
Reviewed By: wez
Differential Revision: D20481051
fbshipit-source-id: 0b0c4d7aea28134383ef45aeafc02930b420286b
Summary:
On Windows, we've had a couple of cases where the overlay is corrupted which
prevent remounting EdenFS. In this case, one entry of a directory was an empty
path. Let's try to catch the error when it happens, instead of after the fact.
Reviewed By: wez
Differential Revision: D21443002
fbshipit-source-id: 47e59aee803cc0b4befd6e0585836d5c68e905b1
Summary:
All of these were simply NOT_IMPLEMENTED on Windows, but the code compiles
and doesn't break any existing tests. The underlying called functions might
have been implemented already, or are NOT_IMPLEMENTED, either way, this reduces
the amount of `#ifdef _WIN32`.
Reviewed By: chadaustin
Differential Revision: D21405622
fbshipit-source-id: bdc2de41d6a57e1c0b532e76eeb2c0c86180d558
Summary:
Only the mtime test had to be disabled due to getMetadata not being tracked
on Windows.
Reviewed By: wez
Differential Revision: D21312680
fbshipit-source-id: 5678956ed2b8b45b38d44c459557161aa6fd222e
Summary:
Fairly straightforward, one change to the test was necessary as reading an
unlinked file cannot be done on Windows.
Reviewed By: wez
Differential Revision: D21312681
fbshipit-source-id: c8cca5eeca7825983176ef618007c514b8e02140
Summary:
This brings it closer to folly::writeFile which should help in avoiding ifdef
whenever we want to use it.
Reviewed By: wez
Differential Revision: D21319020
fbshipit-source-id: 80fbf7fba671b18b5ef68375910e1a2a8869f590
Summary:
The MSVC compiler complains that it doesn't have the full definition of
PrivHelper, causing the build to fail. Include the right header to fix this.
Reviewed By: genevievehelsel
Differential Revision: D21381946
fbshipit-source-id: 0d0389ee8db44a36786973404c38487a94e8c4df
Summary:
While EdenFS does not use a separate privhelper process on Windows, it still
defines a stub PrivHelper class. However, this class was previously defined
in a separate win/utils/Stub.h header file, which led to awkward `#ifdef`s to
include the correct platform-specific header file.
This diff moves the definition of the dummy PrivHelper class in Windows into
the same `PrivHelper.h` header file used on POSIX platforms. This results in
a few more `ifdef`s in the PrivHelper files, but fewer `ifdef`s in the calling
code, and will make it easier to start unifying more of the `EdenMain` logic
on Windows and non-Windows platforms.
Reviewed By: xavierd
Differential Revision: D21332568
fbshipit-source-id: c63bf2b4a8b7e767d7db7dcda28675f735c23bf8
Summary:
FUSE_HANDLE_KILLPRIV expects that any write() call is handled by
clearing the setuid and setgid bits in the userspace. To avoid
implementing that behavior, disallow setting setuid or setgid in the
first place.
Reviewed By: xavierd
Differential Revision: D21333703
fbshipit-source-id: eb084ee8b00afe74c0da26e41c32c2cb742723da
Summary:
Move the `UserInfo` code from `fuse/privhelper` to `utils`, and also unify the
POSIX and Windows implementations of this class.
This code was originally put in the `fuse/privhelper` directory since it was
written at the same time as the privhelper. However, it's really a
lower-level library that doesn't depend on FUSE or any of the other code in
the `fuse/` subdirectory.
Reviewed By: wez
Differential Revision: D21296594
fbshipit-source-id: f58682f6ce86bba0328472c491bb4c0dc3370319
Summary:
The operation originally wanted to operate on the fuse `Attr`
structure which we don't have on Windows, so I repurposed the
`InodeBase::getattr` into `InodeBase::stat` and moved the conversion
of `struct stat` to `Dispatcher::Attr` to the `EdenDispatcher::getattr`
method (and a couple of other adhoc places that were doing a similar
conversion).
Reviewed By: chadaustin
Differential Revision: D20562459
fbshipit-source-id: 6b538110038352e9b5590fcb5ff5c33fe84ac1d8
Summary:
The only change I had to make was due to the fact that MSVC wasn't smart
enough to realize that the shift value couldn't be negative, so a manual
folly::to_unsigned was added to silence the warnings.
Reviewed By: simpkins
Differential Revision: D21268634
fbshipit-source-id: e65f15d58d5ea23bfa6796bab23cf1f5c2e7c12c
Summary:
On Windows, the expected refcount is one less than what it is on Linux, due
to the .eden directory not being present.
Reviewed By: simpkins
Differential Revision: D21268203
fbshipit-source-id: 91cfe742fa4d576917d552964d9541dc68ad2c75
Summary:
In the past, if an EdenMount got stuck while shutting down (inodes still referenced), the EdenMount would already be removed from the server's `mounPoint's` map while stuck. This resulted in `eden list` showing that the mount was not mounted, and then if we tried to remount the mount, we'd get a confusing error message about the not being able to grab the overlay lock. This also made it so if we tried to attach with gdb, or if we core dumped and tried to look at the data, we wouldn't be able to examine the object.
Instead, we should only remove the mount from the `mounPoints_` map after it has completely been shutdown. This will allow us to get better signal on the state of the mount (it will be in the `SHUTTING_DOWN` state in `eden list` instead of `(not mounted)`) and it will allow us to better debug these situations (where a mount is stuck shutting down) since we can get more information about the mount that is in this state.
Reviewed By: simpkins
Differential Revision: D21211544
fbshipit-source-id: 97ff11df67d0a80f47cf6b9f563876e17bd76622
Summary:
From looking at the code, it appears that the FUSE refcount is always a
uint32_t, except when serialized. Let's do the signedness conversion and
narrowing then.
Reviewed By: simpkins
Differential Revision: D21240161
fbshipit-source-id: 877c6cb6881cb36346c64cf92d99b1e588aed580
Summary:
The disabled test uses chmod and symlinks that aren't present in the WIN32
build. I'm not exactly sure why the .eden couldn't be found though.
Reviewed By: simpkins
Differential Revision: D21264070
fbshipit-source-id: da7385d9de4002c99fcbe79f37bd09acd41fbec8
Summary:
At the end of the test, the test assumes that an old uncommited version of
the file can be retrieved thanks to the files living in the overlay. On
Windows, the files are written directly onto the Projected FS cache, and thus
the old version of the file cannot be retrieved.
Reviewed By: pkaush
Differential Revision: D21193209
fbshipit-source-id: a4db6fb2bfc68817601725f97a6d47f2319b83c7
Summary:
While I had to disable a bunch of them at first, this allow them to compile
properly and run. Future diffs will attempt to enable the disabled ones.
Reviewed By: pkaush
Differential Revision: D21188118
fbshipit-source-id: 154fec49c76563b0856fa36e78b2bbd09f0f2381
Summary:
This enables globFiles for Windows, with some
minor tweaks around dtype to enable the build and make
the results consistent between watchman and eden.
Reviewed By: chadaustin
Differential Revision: D20536715
fbshipit-source-id: b1c8184dc664910e4d052a21b4cd993ddfaadf25
