Summary:
Upgrade to the 1.10.0 release. This includes some new features like the
`GTEST_SKIP()` macro.
Reviewed By: genevievehelsel
Differential Revision: D21309360
fbshipit-source-id: 163db628fc99aaa786aeb207f35c7d6295cb5e25
Summary:
Update the generated `run_cmake.py` script to tell ctest to print the test
output on failure. Also pass in a `-j` flag to run tests in parallel by
default.
These flags are already passed in by default when running `getdeps.py test`;
this simply updates this developer utility script to do the same.
Reviewed By: wez
Differential Revision: D21307806
fbshipit-source-id: 42045b0f9362494042c79bc946a1004ff8ad98b6
Summary:
Instead of multiple linear array scans to determine the type of a FUSE
request, use a single lookup table. This is a small optimization, but
it makes the code a bit nicer too.
Reviewed By: wez
Differential Revision: D21314720
fbshipit-source-id: 5b6700ad5bb8e353da1e457de1533e6a626e8f68
Summary:
Google Benchmark is easier to use, has more built-in functionality,
and more accurate default behavior than Folly Benchmark, so switch
EdenFS to use it.;
Reviewed By: simpkins
Differential Revision: D20273672
fbshipit-source-id: c90c49878592620a83d2821ed4bc75c20e599a75
Summary:
Cloning a repository with selectivepull enabled temporarily disables
selectivepull for the duration of the initial pull so that we can get a
streaming clone.
Once this is done, hide all of the commits in the repository by clearing the
visible heads. Selective pull will then populate the remote bookmarks with the
public heads that we do want.
Reviewed By: quark-zju
Differential Revision: D21301037
fbshipit-source-id: 565ae50439ed5405ce940a5675caeba912fe7083
Summary: If a scratch remotebookmark points to a draft commit that is not available locally (i.e., has been hidden on a different machine), then don't pull it into the local repository. Instead, just omit the remote bookmark.
Reviewed By: quark-zju
Differential Revision: D21287886
fbshipit-source-id: 84e7c6b52250709f7c88b07fccdbb61e044370c8
Summary:
Selective pull subscripts are lost during a push. This is because
`remotenames.expush` calls `selectivepullbookmarknames` with the
remote, not the remote name.
`remotenames.exclone` does the same, but in that case there is
no need for the remote name as the destination can't have any
subscriptions other than the default set.
Reviewed By: quark-zju
Differential Revision: D21283029
fbshipit-source-id: 52c2e7e301b8e95b4a252cbfe2ad9de168e81044
Summary:
Commit cloud remote bookmarks sync has several edge cases that don't work correctly.
This test demonstrates some that we will fix.
Reviewed By: quark-zju
Differential Revision: D21283028
fbshipit-source-id: 4e746476a0f3bf0ca7d7088f510451632a2ee075
Summary:
This is a fix after D21067809 broke the mononoke_api unittests and was not caught by sandcastle.
Prior to this change if `ctx.identities` were None then the permission was unconditionally denied, even when the permission checker is set to always_allow. After this change if the identities is None then the permission_checker is called with empty identity set thus making unit tests work properly.
Reviewed By: krallin
Differential Revision: D21324038
fbshipit-source-id: 1edd5fd2af6731d43cab06324b4d9fc02882c09e
Summary:
The systemd binary is installed in a different location on Debian-based
distributions (e.g., Ubuntu) from RedHat-based distributions. Update the
integration tests to look in both places.
Reviewed By: wez
Differential Revision: D21297591
fbshipit-source-id: 67e5a698080205b8624c1f38aa49e75c20b6a28c
Summary:
On Linux the debug info sections in projects downstream from folly and
thrift are huge due to a lot of C++ template usage.
We build in RelWithDebInfo mode as that is most useful when debugging
locally and because the build times are long, but most casual consumers
of the binaries via GH actions really don't care about debug info,
and this should save ~180MB of size in (for example) the watchman
executables.
Pull Request resolved: https://github.com/facebook/watchman/pull/809
Refs: https://github.com/facebook/watchman/issues/804
Test Plan:
Review the artifact size on the linux build in this PR (see
https://github.com/facebook/watchman/actions/runs/91688952) and confirm that it
is a bit smaller than 180MB; now under 3 MB.
Reviewed By: simpkins
Differential Revision: D21318302
Pulled By: wez
fbshipit-source-id: f78bc5e735dd78771e0604abae64c0a23cf9158d
Summary:
Our CMakeLists.txt file requests `ctest` to pass in a `CMAKE_SOURCE_DIR`
environment variable when running our integration tests via `ctest`. However
if a developer manually invokes the test binary this environment variable may
not be set.
For convenience, this updates the code to try looking at the default location
where getdeps will have put the source relative to the build directory (at
least for internal builds using shipit-converted fbsource sources).
Reviewed By: wez
Differential Revision: D21307807
fbshipit-source-id: d747c50d79d2d378721b68b18c256d5363e41f26
Summary:
Update the integration tests to attempt to find the Mercurial config files
from the source directory even in CMake-based builds. This directory will be
available when doing an internal build from fbsource via getdeps.py
If this directory isn't present (e.g., on a build from the github repository)
we will still fall back to using the Mercurial configuration files installed
on the system. In that case I did update the code to use the correct system
configuration path on Windows.
Reviewed By: wez
Differential Revision: D21307808
fbshipit-source-id: 917ba5e8548fed999fb06a29324be125cec83cac
Summary:
Remove the requirement that the `--edenfs` argument flag must be passed in
when invoking the `edenfs` daemon.
This flag was added in D12927803 to help provide a better error message if
users accidentally ran `edenfs` when they really mean to run `edenfsctl`.
However, this shouldn't really be a major problem any more: on Linux and Mac
we now install `edenfs` in a `libexec` subdirectory instead of `bin`, so it
should not be in most user's `$PATH` by default any more. Additionally we
also verify that no other unexpected arguments have been supplied. Prior to
D12927803 the `edenfs` binary would ignore unexpected arguments, making the
error messages more confusing when users tried to run it with `edenfsctl`
subcommand arguments.
One motivation for changing this now is that the Windows version of `edenfs`
does not require this flag, and it is desirable to remove this behavior
discrepancy. Rather than making this flag required on Windows it seems fine
to just drop the requirement on Mac & Linux.
Reviewed By: wez
Differential Revision: D21297159
fbshipit-source-id: e24bd694dadc036cd31dead287ae2c1184747822
Summary:
Update the tests that derive from `ServiceTestCaseBase` to honor the test
blacklist, and to also skip if we cannot run the `fake_edenfs` utility. This
latter check primarily fails when we cannot run `sudo` non-interactively
without a password prompt.
Most of the other tests were already honoring blacklist through the
`EdenTestCase` base class, but `ServiceTestCaseBase` does not derive from
`EdenTestCase`.
Reviewed By: wez
Differential Revision: D21297160
fbshipit-source-id: 5044e9939bbe487c09aa96021166c95f02fb376e
Summary:
Move the `UserInfo` code from `fuse/privhelper` to `utils`, and also unify the
POSIX and Windows implementations of this class.
This code was originally put in the `fuse/privhelper` directory since it was
written at the same time as the privhelper. However, it's really a
lower-level library that doesn't depend on FUSE or any of the other code in
the `fuse/` subdirectory.
Reviewed By: wez
Differential Revision: D21296594
fbshipit-source-id: f58682f6ce86bba0328472c491bb4c0dc3370319
Summary:
Rather than have a single main.yml file containing all off the different
builds, split that up so that we have one file per build environment
(linux, mac, windows).
This has a couple of advantages:
* It is quicker to see the status of just one of the platforms
* Artifact collection for one platform is not blocked pending completion
of the builds for all systems
* It's a little easier to understand what is happening for a single platform
To support having multiple files I've changed the output-file option to
be an output-dir.
I've included the rm of main.yml in this commit. Once this gets
imported back to the FB internal system I'll amend in an update to
the helper script that updates all of our opensource builds and run
and amend that.
Pull Request resolved: https://github.com/facebook/folly/pull/1360
Test Plan:
the GH action status on this PR should show three different
actions running, one for each platform.
I updated and ran
`fbcode/opensource/fbcode_builder/getdeps/facebook/update-all-github-actions.sh`
to regenerate all the actions files for FB.
Reviewed By: yfeldblum
Differential Revision: D21310991
Pulled By: wez
fbshipit-source-id: 604ef652c8f746781a4b410c6b996cdee4524e0d
Summary:
Merge react and userprofile_shell into master, rewrite profile to use react and revert changes to p page
```
Reviewed By: markbt
Differential Revision: D21285297
fbshipit-source-id: f60006738af068982a25fa368fd260ba5b7f3781
Summary:
This commit resolves an issue with our zipapp executables
on Windows that meant that the only reliable way to start them was
to use the fully path to the executable.
The root cause is that the __wargv array is produced by parsing the
process command line into an array, and if you ran `watchman-wait -h`
__wargv[0] would have `watchman-wait` rather than the fully qualified
path to the executable that the zipapp plumbing requires.
The fix is to ask the system for the fully qualified path and ensure that
that gets set as both argv[0] AND argv[1].
Reviewed By: xavierd
Differential Revision: D21190350
fbshipit-source-id: eeb95084592d30a028a93b2b03877f8cc6c72729
Summary:
The environment changed since I tested D20740410 and now it appears
that we'll need to re-export a versioned variable in order for cmake
to detect boost on the GH actions hosts.
Pull Request resolved: https://github.com/facebook/folly/pull/1359
Test Plan:
the GH actions status of this diff.
The workflow was updated via:
```
python3 build/fbcode_builder/getdeps.py generate-github-actions folly --output-file .github/workflows/main.yml
```
Reviewed By: yfeldblum
Differential Revision: D21307640
Pulled By: yfeldblum
fbshipit-source-id: 1555cbcade822775379cd9054be37fdbc17b4d93
Summary: The changes to server/context, gotham_ext and the code that depends on them are the only reminding places where aclchecker is used directly and it is not easy to split this diff to convert them separately.
Reviewed By: krallin
Differential Revision: D21067809
fbshipit-source-id: a041ab141caa6fe6871e1fda6013e33f1f09bc56
Summary:
From the outset, we wanted to be sure that getdeps was able
to source and build the dependencies so that we knew that we'd have
a repeatable build. This came at the cost of build times: having
to build boost on each CI run is a bit of a chore.
This commit adds three new elements to the manifest files:
* `rpms` - a list of RPM names that are all required to be present
in order to consider the dependency satisfied
* `debs` - like `rpms` above, but scoped to debian package names
* `preinstalled.env` - a list of environment variables that if they
are all set and non-empty will satisfy the dependency.
A new `--allow-system-packages` option to getdeps enables the new
logic that looks for system packages; it is off by default, but
enabled in the generated GitHub Actions workflows.
A new `install-system-deps` subcommand is provided that will attempt
to install the system packages needed to satisfy the build. This
typically needs to be run via sudo and is thus broken out separately
from the main getdeps build flow.
I made a pass over the manifest files and added package names that
satisfy the build on ubuntu-18 and fedora-31.
shri-khare: I renamed the `Python3.7.6` manifest to just `python` as
part of this change; the version of python that it pulls in through
the normal build is the same and I believe that an equal or newer
version of python3 is available in the GH actions builder.
The `preinstalled.env` is used only by the boost manifest: it references
the name of an environment variable that is set by the github
windows hosts and that points to a pre-built and pre-installed
copy of boost. Since there is no package manager that we can
easily query for this sort of thing, probing from the environment
seems like a reasonable and fast way to check for this. We
may need to evolve this over time to become more feature rich,
but this seems like a good starting point.
This commit has the potential to save 20 minutes of build time
from each public CI build just due to the boost dependency alone!
Refs: https://github.com/facebook/watchman/pull/797
Reviewed By: yfeldblum
Differential Revision: D20740410
fbshipit-source-id: 6c38019449c54465127656c3d18a6ff1f30adaea
Summary:
Enable the unit tests under eden/fs/utils on Windows.
This does comment out a few tests related to `AbsolutePath` that are broken on
Windows. The AbsolutePath constructor does not enforce that the input path is
actually absolute. Additionally the `canonicalPath()` function ends up doing
weird things and returning paths that start with `/` followed by a drive
letter (e.g., `/C:/foo`). These issues should ideally be addressed in
subsequent diffs.
Reviewed By: xavierd
Differential Revision: D21239886
fbshipit-source-id: ef08d62353ba83b96d9fd79bd4636f4a0f961373
Summary:
When connecting to Mononoke, it's possible for the connection to hang
for a while before ultimately failing. This is a poor experience for users.
Add a timeout, such that hgcli will only wait for 15 seconds before reporting a
failure back to the user.
Reviewed By: krallin
Differential Revision: D21281472
fbshipit-source-id: 6329a108a1ae3ec337d74529a80e0fcd74e02231
Summary: Add a README to provide an overview of the walker for reviewers and maintainers
Reviewed By: ikostia
Differential Revision: D21250280
fbshipit-source-id: 9fd7cd3a076a276de904b8ae5372fc1c0c28458b
Summary:
The operation originally wanted to operate on the fuse `Attr`
structure which we don't have on Windows, so I repurposed the
`InodeBase::getattr` into `InodeBase::stat` and moved the conversion
of `struct stat` to `Dispatcher::Attr` to the `EdenDispatcher::getattr`
method (and a couple of other adhoc places that were doing a similar
conversion).
Reviewed By: chadaustin
Differential Revision: D20562459
fbshipit-source-id: 6b538110038352e9b5590fcb5ff5c33fe84ac1d8
Summary:
The only change I had to make was due to the fact that MSVC wasn't smart
enough to realize that the shift value couldn't be negative, so a manual
folly::to_unsigned was added to silence the warnings.
Reviewed By: simpkins
Differential Revision: D21268634
fbshipit-source-id: e65f15d58d5ea23bfa6796bab23cf1f5c2e7c12c
Summary:
On Windows, the expected refcount is one less than what it is on Linux, due
to the .eden directory not being present.
Reviewed By: simpkins
Differential Revision: D21268203
fbshipit-source-id: 91cfe742fa4d576917d552964d9541dc68ad2c75
Summary: Fix permission issues we are seeing with the latest Mercurial release.
Reviewed By: xavierd
Differential Revision: D21294499
fbshipit-source-id: bcfb13dd005258b2e3b74fa281dbd8df36133ef6
Summary: I thought it would be helpful to introduce and `eden uptime` command, especially with automated graceful restart on the way. This prints it in human readable format, later on if for some reason automation would like to use this, a flag could be added that allows for custom formatting. Also, this can be added to `eden rage` output later.
Reviewed By: chadaustin
Differential Revision: D21260800
fbshipit-source-id: 3f9a4f8d6264dfc38bd15c024a0209f7eeb912fa
Summary:
Previously, for logging the value of unexpected enum values, we wrote
`static_cast<int>`. Given enumerations can have any integral type
backing them, this was somewhat inaccurate. Instead, introduce an
explicit enumValue function which returns a value of the appropriate
underlying type.
Reviewed By: genevievehelsel
Differential Revision: D20975176
fbshipit-source-id: 0bb5b0d2f68f8fe9d68e4c6a847d59ae0997d0df
Summary:
D21074489 adds metrics for pending FUSE requests, this cleans up the display for
pending requests.
This removes the max duration of pending requests for FUSE requests since this
data is not available (it is not measured by the FUSE library).
Reviewed By: chadaustin
Differential Revision: D21074746
fbshipit-source-id: e5585ec091aa5fd5499deee2d8be89f47f769a6a
Summary:
This adds the number and duration of FUSE requests to eden top.
The purpose of this metric is to provide a highlevel indicator of
the health of eden and the fuse interface. Because FUSE is the
interface through which the user mostly interacts with the file
system it these metrics are reflective of edens overall
responsiveness.
The metrics are collected per mount, but we only display the
aggregate since this serves as an overall summary metric.
These metrics are displayed similarly to the import metrics,
for consistency. There is not a metric for the max duration
of pending FUSE requests D21074746 cleans this up so that
the NA is removed from the display.
Reviewed By: chadaustin
Differential Revision: D20928603
fbshipit-source-id: ce78f4274203c2302e7595ae2ef601e18893db38
Summary:
This sets up the counters that will allow us to expose the number
of pending FUSE requests in Eden top.
As D20846826 mentions adding metrics for FUSE request gives
visibility into fuse requests and overall health of eden.
This provides more insight beyond the metrics for live FUSE requests
since it shows the kernels view of FUSE requests. Looking at the difference
between the number of pending and live request, can identify issues
that arise at the interface between eden and FUSE and monitor how
quickly fuse workers are processing requests.
**note**: this is only for linux since macos has no equivalent to
`/sys/fs/fuse/connections`
Reviewed By: chadaustin
Differential Revision: D21074489
fbshipit-source-id: c0951f0dfd4fa764be28d8686d08cd0dd807db37
Summary:
It turns out that macOS, FreeBSD, and Linux all agree on the values
for dtype. If we assume dtype is just the high nibble of mode_t
shifted right by 12, then the Windows CRT agrees too. So hardcode the
values in eden.thrift and add appropriate static_asserts. This opens
the possibility of simply static_cast'ing the numeric values in
Watchman.
Reviewed By: simpkins
Differential Revision: D20975101
fbshipit-source-id: 354ffcbdf3f1d5f8b1715abf1026eaea429d16cf
Summary:
This makes it a bit easier to write integration tests
that run an isolated EdenFS instance complete with its own
unique scratch path environment.
Reviewed By: genevievehelsel
Differential Revision: D21258455
fbshipit-source-id: 3ea13b6337bb8bb58a237e67488b116c423bbe6c
Summary:
In the past, if an EdenMount got stuck while shutting down (inodes still referenced), the EdenMount would already be removed from the server's `mounPoint's` map while stuck. This resulted in `eden list` showing that the mount was not mounted, and then if we tried to remount the mount, we'd get a confusing error message about the not being able to grab the overlay lock. This also made it so if we tried to attach with gdb, or if we core dumped and tried to look at the data, we wouldn't be able to examine the object.
Instead, we should only remove the mount from the `mounPoints_` map after it has completely been shutdown. This will allow us to get better signal on the state of the mount (it will be in the `SHUTTING_DOWN` state in `eden list` instead of `(not mounted)`) and it will allow us to better debug these situations (where a mount is stuck shutting down) since we can get more information about the mount that is in this state.
Reviewed By: simpkins
Differential Revision: D21211544
fbshipit-source-id: 97ff11df67d0a80f47cf6b9f563876e17bd76622
