Commit Graph

2854 Commits

Author SHA1 Message Date
Xavier Deguillard
35ecddf6c0 nfs: add RPC types for READDIR
Summary: This merely adds the types for the READDIR RPC.

Reviewed By: chadaustin

Differential Revision: D26802313

fbshipit-source-id: 634ff9b3f97dc4dba56d225c1fb9eae0a94c02d5
2021-03-18 10:08:50 -07:00
Xavier Deguillard
df4713369e privhelper: disable READDIRPLUS
Summary:
Looking at the spec, READDIRPLUS appears to be more complex to implement than
READDIR, for now, let's force the use of READDIR. Future changes will have to
implement READDIRPLUS as that will likely be a perf improvement.

Reviewed By: chadaustin

Differential Revision: D26802311

fbshipit-source-id: cb784d74507e6c2c2ba4dc0aebe69cfcd69db40b
2021-03-18 10:08:49 -07:00
Xavier Deguillard
ef798f8e3b fuse: rename DirList into FuseDirList
Summary:
This type is very specific to Fuse, let's make it obvious. The readdir method
has also been renamed as it is also very specific to Fuse.

Reviewed By: chadaustin

Differential Revision: D26802309

fbshipit-source-id: c2acdfd1c0006935c59b685fcda729e1bef88928
2021-03-18 10:08:49 -07:00
Xavier Deguillard
9ad3320272 nfs: implement the SYMLINK RPC
Summary:
When creating the .hg directory, Mercurial issues a SYMLINK RPC, thus let's
support it.

Reviewed By: kmancini

Differential Revision: D26785005

fbshipit-source-id: a760d55e6117cc3725444c604e3e4036f4a317b2
2021-03-17 21:30:06 -07:00
Xavier Deguillard
6aa78a0e96 fs: update fuse_kernel_linux.h
Summary:
This is mostly just copying /usr/include/linux/fuse.h from my devserver and
updating some flags in FuseChannel to display the new flags.

Reviewed By: chadaustin

Differential Revision: D27144667

fbshipit-source-id: 4854c6edd4c793ca707db26fecd11e2a3e9d7b75
2021-03-17 20:55:43 -07:00
Katie Mancini
e4f482a144 Fix environment in subprocess calls
Summary:
Some of our subprocess calls are running into dylib errors. The cause looks to
be related to our environment variables. We already have environment hygenics
for buck, so lets borrow this to use elsewhere.

This is to fix prefetch profile fetching on mac, but I ran into another error
when testing `eden du --clean`.

Reviewed By: genevievehelsel

Differential Revision: D27135268

fbshipit-source-id: 3955ddefc5e9ff60e966f63f7dc65ef737186464
2021-03-17 18:59:49 -07:00
generatedunixname89002005307016
1fbe99f5a9 Add annotations to eden/fs/py/test/dirstate_test.py
Reviewed By: xavierd

Differential Revision: D27119152

fbshipit-source-id: 42463371705f4df02445ab4c47fa3b4418c910f0
2021-03-17 14:15:25 -07:00
Zeyi (Rice) Fan
c74f48ed7f sqlite: release StatementCache before releasing SqliteDatabase
Summary: Free the statement cache before trying to release the database. This is causing issues in `eden rm`.

Reviewed By: chadaustin

Differential Revision: D27104645

fbshipit-source-id: 74f4c4e73b048f58b83a28c2d280e5cb1c767048
2021-03-16 23:30:22 -07:00
Anna Kukliansky
d54bb04c4f Handle unused variables
Summary: got this from here: https://stackoverflow.com/questions/1486904/how-do-i-best-silence-a-warning-about-unused-variables

Reviewed By: mzlee

Differential Revision: D27037968

fbshipit-source-id: b5b7ae30fdc192e3d99d3ed857209761a56b5e6f
2021-03-15 23:56:08 -07:00
Anna Kukliansky
b8be7b6b29 fix compilation error of missing type
Summary:
example failure:https://www.internalfb.com/intern/buck/build/89566c50-0f40-4940-bee8-c58e4597db69/

{F508857478}

Reviewed By: mzlee

Differential Revision: D27037961

fbshipit-source-id: dd77219870f368586567b4acf2247cde2855042e
2021-03-15 22:31:25 -07:00
Anna Kukliansky
8da39ce0ae supress deprecated error for INSTANTIATE_TEST_CASE_P
Summary:
suppress the deprecated errors.
found something similar in here: https://stackoverflow.com/questions/1902021/suppressing-is-deprecated-when-using-respondstoselector

example failure:
https://www.internalfb.com/intern/buck/build/a3b550b8-4099-4f27-8975-5bfffd6447e5/

```
eden/fs/inodes/test/OverlayTest.cpp:730:1: error:
'InstantiateTestCase_P_IsDeprecated' is deprecated: INSTANTIATE_TEST_CASE_P is deprecated, please use INSTANTIATE_TEST_SUITE_P [-Werror,-Wdeprecated-declarations]
INSTANTIATE_TEST_CASE_P(
/Users/kuki/fbsource/third-party/googletest/googletest/include/gtest/gtest-param-test.h:507:38:
 note:  expanded from macro 'INSTANTIATE_TEST_CASE_P'
  static_assert(::testing::internal::InstantiateTestCase_P_IsDeprecated(), \
/Users/kuki/fbsource/third-party/googletest/googletest/include/gtest/internal/gtest-internal.h:1209:1:  note: 'InstantiateTestCase_P_IsDeprecated' has been explicitly marked deprecated here
GTEST_INTERNAL_DEPRECATED(
/Users/kuki/fbsource/third-party/googletest/googletest/include/gtest/internal/gtest-port.h:2215:59:
note: expanded from macro 'GTEST_INTERNAL_DEPRECATED'
#define GTEST_INTERNAL_DEPRECATED(message) __attribute__((deprecated(message)))
```

Reviewed By: mzlee

Differential Revision: D27037957

fbshipit-source-id: b12cc500441c9ed4ed72825475c57047fb0c2076
2021-03-15 15:29:12 -07:00
Zeyi (Rice) Fan
af1f5d62ee overlay: use PersistentSqliteStatement in TreeOverlayStore
Summary: This diff replaces `SqliteStatement` in `TreeOverlayStore` with the new `PersistendSqliteStatement`.

Reviewed By: chadaustin

Differential Revision: D26656314

fbshipit-source-id: dbbfeddd52d3be2a1b5d3b6121d3694c41841fd5
2021-03-15 12:01:48 -07:00
Zeyi (Rice) Fan
e79aa979ce overlay: add StatementCache to SqliteOverlay
Summary: This diff adds `StatementCache` to `SqliteOverlay` for reusing prepared `SqliteStatement`.

Reviewed By: chadaustin

Differential Revision: D26656316

fbshipit-source-id: de3fb2bf2a1ec3f057118308287802bd9babcef4
2021-03-15 12:01:48 -07:00
Zeyi (Rice) Fan
89b98ecba6 sqlite: persist statements used in SqliteDatabase
Summary: This diff introduce `StatementCache` for `SqliteDatabase` for caching transaction related SQLite statements.

Reviewed By: chadaustin

Differential Revision: D26656312

fbshipit-source-id: db8b061baa187c9540d1a1a5fcf2f95c3b8a40d0
2021-03-15 12:01:48 -07:00
Zeyi (Rice) Fan
6bba048ea7 sqlite: make SqliteStatement persistent
Summary:
This diff makes `SqliteStatement` to construct persistent statement by default.

This change should be harmless if the caching assumption is not true. If the statement is not cached by the caller, it's probably on a non-critical path.

Reviewed By: chadaustin

Differential Revision: D26639365

fbshipit-source-id: 26af6ccdba036dc0fccf774e9ffb6bb2a014c5ff
2021-03-15 12:01:48 -07:00
Zeyi (Rice) Fan
6162023789 sqlite: split Sqlite.h
Summary: This diff splits Sqlite.h into smaller files so it's easier to change.

Reviewed By: xavierd

Differential Revision: D26656315

fbshipit-source-id: 8156b1d8feef47e5bad4ab66ad5f5f96cf32547e
2021-03-15 12:01:47 -07:00
Zeyi (Rice) Fan
1651dd1898 inodes: implement non-Windows virtual functions for SqliteOverlay
Summary: This diff implements the newly added methods to `IOverlay` so we can build & test `SqliteOverlay` on devserver.

Reviewed By: chadaustin

Differential Revision: D25283647

fbshipit-source-id: 50d0c2dd47cd8f74c860a131ffad1d6342a86ea2
2021-03-15 12:01:47 -07:00
generatedunixname89002005307016
41c3dff77f Add annotations to eden/fs/cli/doctor/test/stale_mount_test.py
Reviewed By: xavierd

Differential Revision: D27038411

fbshipit-source-id: ccaebafebe97767b84981f5c654ce207efed0a87
2021-03-15 09:58:09 -07:00
Katie Mancini
8323ba03c9 allow prefetching windows prefetch profiles
Summary:
Prefetch profiles has been tested to work on windows. Prefetching will be a little slower
on windows as process creation is more expensive, but it is still functional, and should
be ready for us to gather some early numbers on windows sandcastle.

Reviewed By: xavierd

Differential Revision: D27016292

fbshipit-source-id: ed0c8b732b1dfb8bf0a301ae5e76469d4e59f8d2
2021-03-12 17:36:31 -08:00
Chad Austin
7ef5417c64 log a structured event when a proxy hash lookup fails
Summary:
We don't have data on how often this repo-corrupted event occurs, so
begin logging it.

Reviewed By: xavierd

Differential Revision: D26968638

fbshipit-source-id: 95e62a595f74f22fdec8a60ebb0671e53cbe199e
2021-03-12 10:42:46 -08:00
Chad Austin
4b9a93230b optimize HgProxyHash some and make loading from LocalStore explicit
Summary:
It's always annoyed me that HgProxyHash has a constructor which knows
how to load itself from a LocalStore. Add an explicit load() function,
and clean up some other stuff about the class while I'm in there.

Reviewed By: xavierd

Differential Revision: D26769231

fbshipit-source-id: f0ea9f16c3f1fbcd3d4361bcc34845901094b282
2021-03-12 10:42:46 -08:00
Shen Gong
0f0ed89b7c Allow disabling feature Prefetch Profiles
Summary: adds a new subcommand `disable` which will update the user's local eden config with a flag to disable prefetching

Reviewed By: kmancini

Differential Revision: D26964514

fbshipit-source-id: 043c89a5848f827377744ebc46fe93c431efee3e
2021-03-12 09:36:08 -08:00
Xavier Deguillard
3f8ec3cc4c privhelper: tell NFS client to not use locks
Summary:
Since an EdenFS mount can only be mounted in a single place, we don't need to
support the NLM protocol, thus let's inform Linux to not use it.

Reviewed By: kmancini

Differential Revision: D26771237

fbshipit-source-id: bc534a938ad6348bf10a553c6c4f283f043da4f5
2021-03-11 08:31:08 -08:00
Xavier Deguillard
2c1bde2907 nfs: implement the RENAME RPC
Summary:
As it's name implies, this RPC is used to rename files. It's not clear whether
all the error cases that the spec specifies are properly covered, but future
tests can uncover this.

Reviewed By: kmancini

Differential Revision: D26771235

fbshipit-source-id: cad1065a5277e2ab169dd34c7d485d6a4cdd4b76
2021-03-11 08:31:08 -08:00
Xavier Deguillard
9452af6c8c nfs: factor out building the wcc_data
Summary:
This is pattern that is repeated in several functions, let's only have one
function to make the code easier to read.

Reviewed By: kmancini

Differential Revision: D26771236

fbshipit-source-id: 64a68e90eafcea85f850374751ae7bf34f98f118
2021-03-11 08:31:08 -08:00
Xavier Deguillard
6c338a68c6 nfs: add RPC types for RENAME
Summary: This merely adds the types needed for the RENAME RPC.

Reviewed By: kmancini

Differential Revision: D26771238

fbshipit-source-id: 7b9db7b46ffba2d7a906d0e2b60e24df0b5b055d
2021-03-11 08:31:07 -08:00
Xavier Deguillard
30bbc06336 nfs: implement SETATTR RPC
Summary:
The SETATTR RPC allows for changing various attributes of the file, like it's
mode, uid, gid, etc. The one piece of the NFS RFC that isn't implemented is
that NFS allows for a client to pass a ctime to the server that it needs to
check prior to setting the attributes. This is done to avoid concurrent
operations on the file conflicting with each other. This is not implemented for
now as Mercurial appears to not be using it.

Reviewed By: kmancini

Differential Revision: D26760073

fbshipit-source-id: 3474665fcf1b089ef6f7de4a6c45a26ef324240e
2021-03-11 08:31:07 -08:00
Xavier Deguillard
65cc6e6388 Add annotations to eden/fs/cli/config.py
Reviewed By: genevievehelsel

Differential Revision: D26947684

fbshipit-source-id: 53b3af95288788e520582ec7fc1faa3046c064f3
2021-03-10 12:52:48 -08:00
Zeyi (Rice) Fan
b62f870fe8 cli: add debug state for checking projectedfs file state
Summary: A simple debug command to help check ProjectedFS File State on disk

Reviewed By: chadaustin

Differential Revision: D26918172

fbshipit-source-id: 098724fbb0ca1e8eb6798b3ce669a43e2ea576ee
2021-03-10 10:05:42 -08:00
generatedunixname89002005307016
0f3863c783 suppress errors in fbcode/eden - batch 1
Differential Revision: D26914007

fbshipit-source-id: dc68c762e9c33b481bbd98959579e0c0836f17c3
2021-03-09 13:23:39 -08:00
Katie Mancini
ea96dbbc2f make input loop eden top
Summary:
This teaches eden top to read input from the command line which we will use to
switch between help and main page as well as manipulate the process table.

Similar to the python version of eden top q exits. For now we spit back out all
of the other input we get.

Reviewed By: fanzeyi

Differential Revision: D26892622

fbshipit-source-id: 30039d2c2475c036c500c6cb1055c0b2d739691a
2021-03-08 18:03:57 -08:00
Katie Mancini
b3dffd4a54 Add template for rust eden top
Summary:
This adds a subcommand to the rust cli for top command. This sets us up to
start moving over the eden top functionality.

Reviewed By: fanzeyi

Differential Revision: D26892621

fbshipit-source-id: 1d0525291fbad1e5ea97ca22691eacb3fe42ffaf
2021-03-08 18:03:57 -08:00
Xavier Deguillard
f320f5718e config: silence prefetch-profiles config warning
Summary:
On EdenFS startup, it would always print the following warning:

  W0304 16:32:36.103828 642480 EdenConfig.cpp:424] Ignoring unknown section in eden config: /etc/eden/edenfs.rc, key: prefetch-profiles

This is due to that particular config not being specified in EdenConfig.h. By
adding it to EdenConfig.h, the warning disappear.

Reviewed By: genevievehelsel

Differential Revision: D26834504

fbshipit-source-id: 409de118f015226f839cce3ff79a4a2d5b9b43a3
2021-03-05 16:44:48 -08:00
Chad Austin
d45c096063 fast track POSIX ACL xattrs too
Summary:
Fast-track some tricky-to-avoid getxattr requests from the kernel
itself. It would be best to avoid the context switch entirely, but in
lieu of that, at least don't log anything or enter our more expensive
FUSE handling path.

This is follow-up to D24039130 (37f030ba72), which apparently did not completely
eliminate POSIX ACL lookups.

Reviewed By: xavierd

Differential Revision: D26803589

fbshipit-source-id: 18cba8e3ffc45516e6458872e408ed29a562c7a8
2021-03-05 12:29:50 -08:00
Xavier Deguillard
94a46d9c0e inodes: make InodeBase::setattr independant from FUSE
Summary:
Taking a fuse_setattr_in arg means that it can only be used on FUSE, and while
FUSE has been the only backend on UNIX for a while, this is changing with NFS
being added. Thus, we need to find a solution that isn't tied to FUSE.

The solution is fairly simple, let's simply have a struct with optional fields
that needs changing, FUSE and NFS will set these to what needs changing.

Reviewed By: chadaustin

Differential Revision: D26742213

fbshipit-source-id: 16e3e8cdd22d88ace43485d9e3744280de1ee8ad
2021-03-05 11:11:19 -08:00
Xavier Deguillard
8c9839ffcf inodes: return a struct stat from InodeBase::setattr
Summary:
As its name implies FuseDispatcher::Attr is purely FUSE related and thus
shouldn't affect the inodes. It also removes a blocker to using setattr for
NFS.

Reviewed By: chadaustin

Differential Revision: D26742214

fbshipit-source-id: 41a67af0c948d969d5a427f24718de5b134980da
2021-03-05 11:11:18 -08:00
Xavier Deguillard
8a270adec1 nfs: add SYMLINK RPC types
Summary:
This merely adds the types needed for the SYMLINK RPC, the implementation will
follow.

Reviewed By: kmancini

Differential Revision: D26737273

fbshipit-source-id: 4ed3029304fe64892e88bc05a64b4b3b19fd5460
2021-03-05 11:11:18 -08:00
Xavier Deguillard
a579ab83cb nfs: add types for READDIRPLUS
Summary: This is merely adding the type, the implementation will follow.

Reviewed By: kmancini

Differential Revision: D26737271

fbshipit-source-id: 42de7873b271a2bf9499f1274bca50f23dc1016b
2021-03-05 11:11:18 -08:00
Xavier Deguillard
dd831ca0e6 nfs: add XdrList
Summary:
Both READDIR and READDIRPLUS RPC (and other) are using lists as a way to have
an unsized array of values. From a behavior perspective, this is similar to an
array of XdrOptionalValue but without it being prefixed by a size, and with an
additional last element being empty.

To simplify writing these RPC, let's add this XdrList type.

Reviewed By: kmancini

Differential Revision: D26729816

fbshipit-source-id: 8d14bbc6f0513aac51d65625e5751cedc2071a0b
2021-03-05 11:11:18 -08:00
Xavier Deguillard
3371db5c25 nfs: move the XDR macros to Xdr.h
Summary:
These aren't tied to the RPC protocol but more to how XDR works, therefore it
makes more sense to move these to the Xdr.h file. It also enables tests to use
it instead of having to write the serialization by hand.

Reviewed By: chadaustin

Differential Revision: D26729815

fbshipit-source-id: 227299b94e2f6fa5ecc747a8d5705f0931a7d1f8
2021-03-04 18:15:33 -08:00
Xavier Deguillard
ba37453611 nfs: add RPC types for SETATTR
Summary: This merely adds the types for the SETATTR RPC.

Reviewed By: kmancini

Differential Revision: D26707362

fbshipit-source-id: 89cbf7a501cf4e13878cd84b6c36691d2a08594f
2021-03-04 18:15:33 -08:00
Xavier Deguillard
aed51e9e44 nfs: add argument formatting
Summary:
Now that a large part of the RPC is implemented, being able to see what
arguments are being passed in makes it convenient for debugging.

Reviewed By: kmancini

Differential Revision: D26707363

fbshipit-source-id: 676adef996bf584140b324df1419b139f4600c91
2021-03-04 18:15:33 -08:00
Xavier Deguillard
72972c0ff5 nfs: implement the READ RPC
Summary:
This reads the data from the file. One of the optimization that isn't done in
this diff would be to detect the EOF without having read return a 0-bytes long
buffer. For now this isn't done, but more plumbing in the FileInode would be
required to achieve this.

Reviewed By: kmancini

Differential Revision: D26704138

fbshipit-source-id: 82e6d4994dd5dc46372778c10d9f68f7ca1e3a87
2021-03-04 17:51:51 -08:00
Xavier Deguillard
6629d7fd9b nfs: add RPC types for READ
Summary: This merely adds the types for the READ RPC.

Reviewed By: kmancini

Differential Revision: D26704137

fbshipit-source-id: a803fc9082cbec6ce0c6b9db2bfede55bf478df1
2021-03-04 17:51:51 -08:00
Xavier Deguillard
7238290100 nfs: implement the REMOVE RPC
Summary:
While creating the .hg directory, Mercurial seems to want to remove some files
too, thus let's thread this to the inode layer to allow Mercurial to continue.

Reviewed By: kmancini

Differential Revision: D26699272

fbshipit-source-id: e6d283b92e18962698c0df3c6361adfbd0c50223
2021-03-04 17:51:51 -08:00
Xavier Deguillard
4707bc22ec inodes: thread an ObjectContext into unlink/rmdir
Summary:
All the callers of these methods have a real context in hand, let's use it
instead of creating a null one.

Reviewed By: genevievehelsel

Differential Revision: D26699271

fbshipit-source-id: 9fb268c9b3194d7e951e30ab5e90c4f2f0388e81
2021-03-04 17:51:50 -08:00
Xavier Deguillard
9a7c51ef9c nfs: add RPC types for REMOVE procedure
Summary: This is merely adding the types needed to implement the REMOVE procedure.

Reviewed By: kmancini

Differential Revision: D26694441

fbshipit-source-id: 736d06ed7bfae40f827fbec6017e069fe11d7464
2021-03-04 09:59:48 -08:00
Xavier Deguillard
6daea7e4e0 nfs: implement the LINK RPC
Summary:
Since EdenFS doesn't support hardlinks, this is merely collecting the attribute
of the file before always failing.

Reviewed By: kmancini

Differential Revision: D26682014

fbshipit-source-id: 8c39170147023a03e87ed7a0fae66f9184b4f690
2021-03-04 09:59:47 -08:00
Xavier Deguillard
466b3144b8 nfs: add RPC types for LINK
Summary: During clone, Mercurial calls this, despite EdenFS not advertising hardlinks.

Reviewed By: kmancini

Differential Revision: D26682013

fbshipit-source-id: b43affeb7577354991308165113d5d5070bf8605
2021-03-04 09:59:47 -08:00
Xavier Deguillard
b5ad70b619 nfs: implement the FSSTAT RPC
Summary:
The RPC simply queries various filesystem attributes, we merely forward what
statfs on the overlayfs gives us.

Reviewed By: kmancini

Differential Revision: D26681613

fbshipit-source-id: 5b94d05cafff8d77390fe60a3b5cf1dc3e022f42
2021-03-04 09:59:47 -08:00