Summary: sys.platform is "win32", not the path to python.
Reviewed By: fanzeyi
Differential Revision: D31554106
fbshipit-source-id: 64b388d2fb8a493f811a0cf22fe2471a25bfbf7e
Summary: This appears to have broken all the tests on Linux.
Reviewed By: zhengchaol
Differential Revision: D31505082
fbshipit-source-id: 610eb941d0f0eb536a4688ac2637a8466be0b05c
Summary:
On Windows, edenfsctl.real is a par file that Windows can't execute directly.
Thus let's have Python run it.
Reviewed By: fanzeyi
Differential Revision: D31477721
fbshipit-source-id: d5a699ceb3d3b1b3d5778ef5720bca7c299bed80
Summary:
These tests have been failing for a long time due to the expected output on
Windows containing \ but the test using the / separator.
Let's simply use os.path.join to build a path with the right separator in these
tests.
Reviewed By: fanzeyi
Differential Revision: D31477722
fbshipit-source-id: a4ac25750647229974e23e305508e83917011fef
Summary:
The goal of this stack is to remove Proxy Hash type, but to achieve that we need first to address some tech debt in Eden codebase.
For the long time EdenFs had single Hash type that was used for many different use cases.
One of major uses for Hash type is identifies internal EdenFs objects such as blobs, trees, and others.
We seem to reach agreement that we need a different type for those identifiers, so we introduce separate ObjectId type in this diff to denote new identifier type and replace _some_ usage of Hash with ObjectId.
We still retain original Hash type for other use cases.
Roughly speaking, this is how this diff separates between Hash and ObjectId:
**ObjectId**:
* Everything that is stored in local store(blobs, trees, commits)
**Hash20**:
* Explicit hashes(Sha1 of the blob)
* Hg identifiers: manifest id and blob hg ig
For now, in this diff ObjectId has exactly same content as Hash, but this will change in the future diffs. Doing this way allows to keep diff size manageable, while migrating to new ObjectId right away would produce insanely large diff that would be both hard to make and review.
There are few more things that needs to be done before we can get to the meat of removing proxy hashes:
1) Replace include Hash.h with ObjectId.h where needed
2) Remove Hash type, explicitly rename rest of Hash usages to Hash20
3) Modify content of ObjectId to support new use cases
4) Modify serialized metadata and possibly other places that assume ObjectId size is fixed and equal to Hash20 size
Reviewed By: chadaustin
Differential Revision: D31316477
fbshipit-source-id: 0d5e4460a461bcaac6b9fd884517e129aeaf4baf
Summary: This diff fixes some integration test errors after enabling the new edenfsctl.
Reviewed By: xavierd
Differential Revision: D30789741
fbshipit-source-id: 02d74defc41def4fb6ea0cc4694f944b4c0044e2
Summary: This diff adds the Rust edenfsctl to our open source build.
Reviewed By: xavierd
Differential Revision: D30788685
fbshipit-source-id: 603caa933ecfc5af0ede7e22f6c7911462da3a65
Summary:
We run periodic inode unloading for unlinked inodes on NFS because we get no
information from the client on when inodes are no longer needed, and we have to
clean them up at some point for memory and disk reasons. See previous commit
summaries for more details on this (D30144901 (ffa558bf84)).
Let's add some counters on this so we have a bit more visibility into the
process. This counter is meant to mimic the PeriodicUnloadCounter counter.
Reviewed By: chadaustin
Differential Revision: D30966688
fbshipit-source-id: cfc8d769b53073d9f4c0c27b6bee20e222c6c8d2
Summary:
It can be surprising when a job suddenly is no longer able to run sudo, or no
longer run as root that all the tests are marked as being successful, despite
the fact that they no longer run. Let's recognize when we run on EdenFS to
allow tests to fail if they can no longer run EdenFS.
Reviewed By: zhengchaol
Differential Revision: D30357402
fbshipit-source-id: c3758d7a5a3c575dd68bd97062ae24abe4124874
Summary:
Now that we might have multiple kernel protocols per mount (i.e. both fuse and
nfs on macOS) let's include them in eden rage.
Reviewed By: xavierd
Differential Revision: D31154042
fbshipit-source-id: 38e7630829d70fe9dd6dbeabacc3b538ee798e0d
Summary:
We periodically need to dereference inodes on NFS because we get no other info
from the kernel on when should dereference them.
This means the NFS kernel might have references to inodes after we delete them.
An unknown inode number is not a bug on NFS. It's just stale, so the error should
reflect that.
Reviewed By: xavierd
Differential Revision: D30144898
fbshipit-source-id: 3d448e94aea5acb02908ea443bcf3adae80eb975
Summary:
`eden prefetch` and `eden glob` return lists that despite being called
"maching files" actually contains both files and directories.
In some cases, we only want the list of files and it introduces unnessecary
overhead on our clients for them to have to stat all the files in the list to
filter out the dirs. Let's add an option to just list files.
Reviewed By: chadaustin
Differential Revision: D30816193
fbshipit-source-id: 6e264142162ce03e560c969a0c0dbbc2f418d7b9
Summary: the rage summary is getting hard to quickly parse, so this underlines each section header, as well as unifies underline looks (with `eden stats`). This adopts the underline code from `eden du` and makes it a util function for shareability.
Differential Revision: D30857773
fbshipit-source-id: 66b5b06f5b0125304d45d3465a8bc2248693b791
Summary: This test can fail for not having `USERNAME` environment variable when the test is run as root. Let's just skip the test when this happens because it doesn't make sense to drop priv as root.
Reviewed By: xavierd
Differential Revision: D30868518
fbshipit-source-id: 14ff6db218b1477f5905f2df3ad075a5ca186117
Summary:
This change has the unintended effect of causing any Thrift calls to
potentially issue a recursive EdenFS call due to symlink resolution requiring
running `readlink` on the root of the repo itself.
Fixing this isn't really possible, thus let's revert the change altogether, we
can force clients to issue a realpath before issuing EdenFS Thrift calls.
Reviewed By: kmancini
Differential Revision: D30550796
fbshipit-source-id: 9494c8e08c8af2392eeb344879f156cb56f93ea6
Summary:
The rate metric can be unreliable, and in some environment is never updated by
the time it is read by the test. Since this test cares about the number of
events, using count is a better metric, and more reliable.
Differential Revision: D30377128
fbshipit-source-id: f10c656567e3a3c07b66ecc6fc563a53199e3088
Summary:
In the ASIC test environment, we've see cases where this isn't set, let's
simply skip the test in this case instead of failing.
Reviewed By: chadaustin
Differential Revision: D30353929
fbshipit-source-id: 956da6f8f12de025b8ca72e40097f1f9d50e6bf7
Summary:
In the case where the path to the mount has symlinks, EdenFS would only accept
the path to it that was specified at mount time, even though another path may
refer to the same directory.
To solve this, we can simply normalize paths in all the Thrift endpoint to make
sure that EdenFS always refers to a mount point under its non-symlinked path.
Reviewed By: chadaustin
Differential Revision: D30320515
fbshipit-source-id: e578d059a3b1307d6b24c4b9bdb1ceb3b534c460
Summary:
The changes in this diff makes the code to "run" and skip correctly according to our rules.
All integration tests are now runnable on Windows with mode/win
Differential Revision: D30143255
fbshipit-source-id: b2ddbff7268f182274b3755f4b28df6ac6cdeef4
Summary:
This newly added config controls the default behavior of `eden clone` and
whether a new clone will use NFS or FUSE. This is intended to facilitate the
transition to NFS from FUSE on macOS.
Differential Revision: D30110056
fbshipit-source-id: ea6b493aa803297952b46434f6d11d8edf58e40b
Summary:
This adds the options to `eden stats` for collecting only fast stats and printing in JSON.
`eden stats` can be slow especially due to collecting fb303 counters and private bytes. An example use case of this new lightweight endpoint is that Buck can poll it to display Eden related info in its cli (see [post](https://fb.workplace.com/groups/132499338763090/permalink/210396380973385/) for context).
Reviewed By: xavierd
Differential Revision: D29687041
fbshipit-source-id: a663e71231527c5dfb822acbf238af0ac6ce4a00
Summary:
Per comment in D30017261 (e9c039ab4a)
> [...] we can get rid of this ESTALE check. That dates back to when edenfs had a bug that returned ESTALE when reading the .eden directory sometimes.
Reviewed By: xavierd
Differential Revision: D30024979
fbshipit-source-id: 645097c8f689c916245845561fd3d824ff7df8b4
Summary: 2 of the disabled are passing, and the last one just needed a simple tweak.
Reviewed By: fanzeyi
Differential Revision: D30029260
fbshipit-source-id: ad386632e8b6fbf9c76530c8ce2df7d38990bbbd
Summary:
The mode bits don't make a lot of sense on Windows, so let's use the plain
0o777 as that's what they will look like on Windows.
Reviewed By: chadaustin
Differential Revision: D30028892
fbshipit-source-id: ced737f1290438708536b732095cee18e1b1c073
Summary: The test is passing, enable it.
Reviewed By: chadaustin
Differential Revision: D30028894
fbshipit-source-id: b268fc63220704466c72c688a1f0f04a47573641
Summary:
The create_editor_that_writes_commit_messages function uses bash which is not
available on Windows. Instead, we can simply pass --message to Mercurial for
the same effect.
Reviewed By: chadaustin
Differential Revision: D30028893
fbshipit-source-id: 0a588c37bd4da85176546568126893192b29a574
Summary:
If Mercurial asks EdenFS to update to a commit that it has just created, this
can cause a long delay while EdenFS tries to import the commit.
EdenFS needs to resolve the commit to a root manifest. It does this via the
import helper, but the import helper won't know about the commit until it is
restarted, which takes a long time.
To fix this, we add an optional "root manifest" parameter to the checkout or
reset parents thrift calls. This allows the Mercurial client to inform EdenFS
of the root manifest that it already knows about, allowing EdenFS to skip this
step.
Reviewed By: chadaustin
Differential Revision: D29845604
fbshipit-source-id: 61736d84971cd2dd9a8fdaa29a1578386246e4bf
Summary:
We need to set the infinitepush path too so that the Mercurial autopull code
won't try to pull from Mononoke
Reviewed By: genevievehelsel
Differential Revision: D29943227
fbshipit-source-id: a67dbfe97e7ab46dee885d9ec91a4d194dc2bd37
Summary:
If for some reason EdenFS cannot be started, we shouldn't attempt to run the
fsck tests as these would always fail.
Reviewed By: genevievehelsel
Differential Revision: D29918436
fbshipit-source-id: 6e4a01a747157427e5c1028084e32cef8066c96a
Summary: These are needed for mercurial in the test cases, we set this in the testharness C++ repo as well
Differential Revision: D29868460
fbshipit-source-id: e11cf41823ee073e3863fb5a38ecbf1146073ff5
Summary: This change let Eden cli can ```clone``` and ```info``` on a RE Digest backed store
Reviewed By: chadaustin
Differential Revision: D28855458
fbshipit-source-id: 5582992acc5b3b3acb05b0b53d59a6768cc02491
Summary: Currently the tests manually run fsck command (Python) on the snapshot. This makes the change so that the tests will not only test the manual fsck command but also test the auto fsck (C++) so that we cover both Eden CLI fsck (Python) and Eden mount auto fsck (C++).
Reviewed By: chadaustin
Differential Revision: D29690188
fbshipit-source-id: 593db1db587c3294aad1314ea8da1d8e778df8ee
Summary:
This test is broken due to Rust panic from hg (likely there was some change in hg and the original snapshot was from 2018). It's hard to know exactly what's causing this issue because 1) it's not from eden 2) this has been broken for a while so it's tricky to bisect.
To bring back the coverage and run on a modern repo, this diff:
* generates a new snapshot (`buck run //eden/integration/snapshot:gen_snapshot -- basic`).
* adds a step to translate hgrc because the path at snapshot generation time is in it.
* migrates this test to the new snapshot
Reviewed By: kmancini
Differential Revision: D29670241
fbshipit-source-id: 1c5dccc673d516334de83582b32e2e3c9dc308f1
Summary: The hybrid changelog relies on edenapi which isn't currently supported by tests. Disable the migration for integration tests, until test repos are able to use edenapi.
Reviewed By: singhsrb
Differential Revision: D29602284
fbshipit-source-id: 8a2b4395fc5717c3880d7b74c45a0aef571cdc17
Summary: There are a lot of places in user visible text such as command line help where EdenFS is mentioned as eden/Eden/edenfs/EdenFS. This will make it consistent to 'EdenFS' in most cases. In the places where it is referring to the process/daemon, 'edenfs' will be used.
Reviewed By: chadaustin
Differential Revision: D29419151
fbshipit-source-id: 7b8296f0a0c84fdcb566ff811f7fcedbe7079189
Summary:
Like in many of the other cases, this needs to be told where the Eden binary
is instead of assuming it's right next to the edenfsctl binary, because on Buck
2 it's not.
Reviewed By: xavierd
Differential Revision: D29265845
fbshipit-source-id: 756bd863dc7d18eaf25a9ee209a9fd59345e6b5d
Summary:
This assumes the Eden binary is next to edenfsctl. Like in the previous diffs,
it isn't.
Reviewed By: xavierd
Differential Revision: D29200149
fbshipit-source-id: a7d335c5f3748c795b20480e26b24b9287708c50
Summary:
Like it says in the title. This fails because it assumes it's next to the
EdenFS binary but that's not true in Buck v2.
Reviewed By: xavierd
Differential Revision: D29200150
fbshipit-source-id: 1be5b6c647047a50dd553031691e9bbec5f9caaf
Summary: Pretty much the same as D29061439 (8f263485d9), but in another place that needs it.
Reviewed By: xavierd
Differential Revision: D29197979
fbshipit-source-id: eafccefee54a64131eb7547ad19ab1da708852e6
Summary:
On case insensitive mounts, updating between commits that have a different
casing for a file/directory would lead to the update failing due to EdenFS
believing that an untracked file is present in the mount. That conflict is
however bogus and EdenFS simply gets confused in
TreeInode::processCheckoutEntry about the entry with the different casing.
To fix this, we should avoid comparing paths in a case sensitive manner and
instead compare then in a case insensitive fashion. This allows the rest of the
checkout code to update the directory/file in place.
On Windows, there is one more subtlety: we can't change the casing of a
file/directory that is already a placeholder and thus we need to force the
entire hierarchy to be removed, this will also make the checkout fail in case
of untracked files in the hierarchy, which is also the behavior on case
sensitive systems.
Reviewed By: genevievehelsel
Differential Revision: D29121741
fbshipit-source-id: 3d2cdacf296a3d061fc828cd6d04d249542cb63f
