Commit Graph

3256 Commits

Author SHA1 Message Date
Adam Simpkins
1de7c09d61 fix a crash in GlobNode::evaluate()
Summary:
Fix `GlobNode::evaluate()` so that it only generates a result once all
processing is complete.  The glob evaluation walks down the inode tree in
parallel.  Previously if an error occurred while processing one TreeInode it
would return that exception to the caller immediately, even if work was still
being done to evaluate the glob on other parts of the inode tree.  This
behavior is problematic, since the caller may destroy the GlobNode immediately
once a result is generated, and the caller has no way to know that other
children GlobNodes may still be using some of the data.

This fixes the code to wait until all evaluation sub-futures have completed
processing even on error, so that we know it is safe to destroy the GlobNode
before we generate a result.

Reviewed By: wez

Differential Revision: D17434327

fbshipit-source-id: ef70dc82281eeb3540840202ec5e704afe2d29eb
2019-09-17 16:23:56 -07:00
svcscm
63a82fb878 Updating submodules
Summary:
GitHub commits:

da3af3e555

Reviewed By: yns88

fbshipit-source-id: 4b6152e1073bceda3137b4fd7ccccc1ed6482771
2019-09-17 15:45:38 -07:00
Wez Furlong
eeb57507e4 eden: du: ignore FileNotFoundError when stat'ing ignored files
Summary:
It's possible that status includes ignored files from
the overlay that are no longer accessible because a bind mount
is overlaid on top of the overlay!

Ideally we'd account for the space occupied by those files,
but for now we'd rather not error out when attempting to stat
them, so silently ignore that class of error.

Reviewed By: chadaustin

Differential Revision: D17415965

fbshipit-source-id: 96d467ee4e85ad5bdc730e4cfffe4c15a83a12ed
2019-09-17 15:03:23 -07:00
svcscm
6d0f6fb879 Updating submodules
Summary:
GitHub commits:

f7d6a31703

Reviewed By: yns88

fbshipit-source-id: d0e7a89e985152aea2e2b6a4c70ac9c8f6fb20c1
2019-09-17 12:43:35 -07:00
Wez Furlong
51eaad1918 eden: chown now accepts user and group names (not just ids)
Summary: This makes it a bit more human friendly

Reviewed By: chadaustin

Differential Revision: D17249465

fbshipit-source-id: 40d5afc77ded34237e1860d5b91e9257a732e480
2019-09-17 07:29:40 -07:00
Wez Furlong
d9942f41ce eden: call hg update hooks at clone time
Summary:
Some of our internal repos have important post-update
hooks and we currently are not calling them for the initial clone,
so do that.

Reviewed By: chadaustin

Differential Revision: D17244812

fbshipit-source-id: 3424458710a08bf3bcd3331a09a643a90a6a63be
2019-09-17 07:29:40 -07:00
Wez Furlong
b2d7ab7fec eden: teach chown to use redirection data
Summary:
D17236366 will disable the getBindsMount thrift call and
remove the internal source of data about bind mounts.  We instead
have a more current set of data from the `redirect` command, so
tech `eden chown` to use that data.

Reviewed By: chadaustin

Differential Revision: D17249433

fbshipit-source-id: 853f24e729814c501768e9834765e1be283d6aac
2019-09-17 07:29:39 -07:00
svcscm
6f0b91c8f5 Updating submodules
Summary:
GitHub commits:

aa2b3df92d
4b12595724

Reviewed By: yns88

fbshipit-source-id: 5ae3517d4795492f3aa0915540475fb3dd10a8ab
2019-09-17 07:29:39 -07:00
svcscm
047a095585 Updating submodules
Summary:
GitHub commits:

b2ad9c27cd
774ef72134
e01c5188d5
464a4b57db
6287f0d73b
7859497700
c90c863685
48892814e2
6405617405
9036b9e85f

Reviewed By: yns88

fbshipit-source-id: 5fc47df704471e1b99649223c168fd602d555e48
2019-09-16 21:29:05 -07:00
Adam Simpkins
e286e4c888 fix the thrift CMake rules to add dependencies on the thrift compiler
Summary:
Update the thrift C++ and Python CMake rules to indicate that the output also
depends on the thrift compiler itself.

Previously the C++ rule indicated that the output depended on the thrift
template files, which caught most cases when the thrift compiler was updated,
but wasn't fully correct.  The thrift templates were also removed and baked
into the thrift compiler binary in D16356056.

Reviewed By: yfeldblum, chadaustin

Differential Revision: D17401217

fbshipit-source-id: ae5cde7a7e5e07a74406a1b6f4469124187bc12f
2019-09-16 21:10:01 -07:00
Adam Simpkins
d7fbfd8479 add a command line option to disable the build cache
Summary:
This is useful when working on changes to some of the builder functions,
to skip ever trying to use cached results.

Reviewed By: chadaustin

Differential Revision: D17401219

fbshipit-source-id: fb7d5ea45618653957b9bd6a62eed91d8334824d
2019-09-16 21:10:01 -07:00
svcscm
3e21fbbd32 Updating submodules
Summary:
GitHub commits:

22c06084b8
5313a89c38
18836fbae0
81217e50a5
4c993e5dc0
0d6d85ff26
ba1537d7a4
c0ad1625c8

Reviewed By: yns88

fbshipit-source-id: d0cd56fbf45e07b689220fb975c481d9c1a314b6
2019-09-16 21:10:01 -07:00
svcscm
7b2a2ac48a Updating submodules
Summary:
GitHub commits:

d12c28952d
a09ffe23e7
9dc9ee901b
f3531c5fa5
74efa0633d
ce83620636
5040e7f288
af51be5bcb
549ea0a864

Reviewed By: yns88

fbshipit-source-id: 4d7cbe140667ba738af5e802135efbdd9f12c396
2019-09-16 17:59:52 -07:00
Wez Furlong
2bde83ebd9 getdeps: adjust fixup-dyn-deps to optionally use absolute paths
Summary:
This commit teaches fixup-dyn-deps how to generate correct
absolute paths in the context of the ultimate install path (specified
via the `--final-install-prefix` option)

Absolute paths are desirable if you have, for example, an executable
that you wish to install with the setuid bit set.

Reviewed By: simpkins

Differential Revision: D17361491

fbshipit-source-id: 4c4f3f15208266300622f84dc9cd1ba83883dfb7
2019-09-16 17:37:57 -07:00
svcscm
939ab534f2 Updating submodules
Summary:
GitHub commits:

3f60295979
ef0eb1eef5
0009b19189
68626249c3
79f4abaa86
cfa482e889
5602c2ad9d
2314bbaf7d
d49dcc4445

Reviewed By: yns88

fbshipit-source-id: 7e6ace9d7322bb23e89a95d5162fc81ac23a8e6c
2019-09-16 17:37:57 -07:00
svcscm
e0887a06e1 Updating submodules
Summary:
GitHub commits:

c6972d0856
ef8a9b3830
f3e3ab0ff5
a783cdd351
7767a193cb
638d239507
9559cf04a2
268594315b
5ce193a796
4694a6908e
1b53d53a07

Reviewed By: yns88

fbshipit-source-id: 1970f397164c0d80351236691b3a51a58332efe3
2019-09-16 16:08:46 -07:00
Chad Austin
23363ac68e fbcode_builder: add a license header to FBBuildOptions.cmake
Summary:
Add a license header to satisfy the open source linter. Use the same
header the other .cmake files have.

Reviewed By: mhlakhani

Differential Revision: D17404782

fbshipit-source-id: 66679d72c9e680f8bb8b27869e981a046b3520cf
2019-09-16 15:41:57 -07:00
svcscm
0193f5cef4 Updating submodules
Summary:
GitHub commits:

b737304fcc
8075e22543
a4bcc2e441
140ffd3989
5d7bf4a4cf
66648e3741
24c18300d1
2c34cc2568

Reviewed By: yns88

fbshipit-source-id: 11036e658b052d866459b99b964fa7060f9fa437
2019-09-16 13:45:12 -07:00
Chad Austin
a4311562e4 build integration test helpers with open source gflags
Summary:
Integration test helpers relied on an implicit gflags include. Make
that explicit so they compile against open source gflags and glog.

Reviewed By: wez

Differential Revision: D17264335

fbshipit-source-id: e336423b71c0f15e29b0e4ad604328b7624080a8
2019-09-16 13:32:15 -07:00
svcscm
27dfd8a7f0 Updating submodules
Summary:
GitHub commits:

c18b4a1324
bff2c74800
a92b821db2
351071d9cd
1a768a3962
9eb450d573
77860f0a62
0d92268cbe
7ee1bffda2

Reviewed By: yns88

fbshipit-source-id: 66b0c966b579ac0f6a688068f355254ac4deb679
2019-09-16 13:27:34 -07:00
Wez Furlong
31d7348f33 Let's try a github action to build things (#743)
Summary:
This commit adds a getdeps command that is able to generate
a workflow file for the GitHub Actions CI environment.

The workflow file could be expressed more simply using the matrix
syntax and with three steps (checkout, build, test), but I chose to
break out the steps for each of the dependencies because the UX
while waiting on the build is much nicer that way: the steps show
during and live log tailing for the section of the build that is
underway.  If they were all lumped into a single build step then
the logs from the boost section of the build dominate and make
the github UI work very hard.

Pull Request resolved: https://github.com/facebook/watchman/pull/743

Test Plan:
https://github.com/facebook/watchman/pull/743 successfully
executes the github actions CI flow.

```
$ opensource/fbcode_builder/getdeps.py generate-github-actions --output-file watchman/.github/workflows/main.yml watchman
```

Reviewed By: simpkins

Differential Revision: D17384915

Pulled By: wez

fbshipit-source-id: 9a9e5a3e806c18f6cc38ba1cb7059740cda01ad4
2019-09-16 12:58:19 -07:00
Wez Furlong
cae23d219b getdeps: teach builder how to find vs 2019
Summary:
GitHub Actions CI `windows-latest` environment has only VS 2019
installed, so we need to expand our logic to be able to locate it.

Note that Boost 1.69 doesn't know how to locate VS 2019 so we are effectively
tied to VS 2017 at the moment; the search order in this diff reflects that.

(That means that we can't target `windows-latest` on GitHub Actions, but that
is really a concern for a later diff in this stack)

Reviewed By: simpkins

Differential Revision: D17385052

fbshipit-source-id: 9bb0612154f42d425a625406488f39bb4ec3d8ae
2019-09-16 12:58:19 -07:00
Wez Furlong
4a64d1f60d getdeps: prefer MSVC over gcc when building on GitHub Actions CI
Summary:
while testing https://github.com/facebook/watchman/pull/743 I
noticed that the cmake builds were picking up the installed mingw GCC
compiler rather than the MSVC compiler.  That would be fine except that
boost is built with MSVC and its generated libraries cannot be subsequently
found by a cmake gcc build that uses FindBoost.

This commit forces cmake to pick cl.exe rather than gcc.  This is probably
fine to do unconditionally on windows, but since I've only observed this
particular problem with GitHub Actions I'm keeping it constrained to that
environment for now.

Reviewed By: simpkins

Differential Revision: D17385050

fbshipit-source-id: 90bef898b138e5d4bbd28a155ed1bd468acee9de
2019-09-16 12:58:19 -07:00
Wez Furlong
935867f17e getdeps: restructure bison+flex dependencies
Summary:
We've been squeaking by with assuming that flex is installed already
on posix systems, but that isn't the case on the github actions default
configuration.

Adjust the bison recipe: on windows it deploys both flex and bison.  We use the
same source for both flex and bison but install flex to a separate install
prefix to make it easier to consume the flex dependency distinct from the bison
dependency.

The latest flex release segfaults during compilation on linux unless we
force -DGNU_SOURCE, so the manifest does that on linux.

Reviewed By: simpkins

Differential Revision: D17385051

fbshipit-source-id: 9f31b07849af9de50099d1b20bedba517bbbdf2f
2019-09-16 12:58:19 -07:00
Wez Furlong
538643c226 getdeps: fix openssl dep for libevent on macos
Summary:
while testing https://github.com/facebook/watchman/pull/743 on macOS
I noticed that the libevent build failed to find openssl.

openssl is special on macos because apple do not ship the headers.
We already build and depend on openssl for the folly manifest on
macos, so this is really just adding a missing edge.

Reviewed By: simpkins

Differential Revision: D17385053

fbshipit-source-id: 1b688537fef422d81a959fc5749c871b9e868baa
2019-09-16 12:58:18 -07:00
svcscm
dadddc724b Updating submodules
Summary:
GitHub commits:

8271eadc36
0cc120cf18
956efeecd4
9a90f3e996

Reviewed By: yns88

fbshipit-source-id: f171cd156ffbaff6f2bc1fe8e15ad8df1f9a4a06
2019-09-16 12:58:18 -07:00
svcscm
e93362d361 Updating submodules
Summary:
GitHub commits:

09d4c888f5
915d72d849

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 46435ff8e118c817f052d27ed67bfc6336629855
2019-09-16 12:21:29 -07:00
svcscm
742348c44f Updating submodules
Summary:
GitHub commits:

d44f5f8be6

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: a1dad2eca9ad62c3751349775c2775ba8ae2ff4c
2019-09-15 17:22:02 -07:00
svcscm
d74561e7b6 Updating submodules
Summary:
GitHub commits:

f12ca9330a

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 5cc801f85f48361af364c114a9f8d0e1d233cf88
2019-09-14 16:40:32 -07:00
Wez Furlong
4099e9c561 eden: teach systemd not to use sudo if installed setuid
Summary:
Building on D17288111, this allows the systemd integration
to start eden without using sudo.

Reviewed By: chadaustin

Differential Revision: D17362519

fbshipit-source-id: 4a18380ff8de02e659bf4f906ecfffbf25ce7ad7
2019-09-13 18:38:23 -07:00
svcscm
03c2833514 Updating submodules
Summary:
GitHub commits:

e43fac0667
ebec93cbac

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 6a3fd35c4a8f183541300cb0c4763ab073acfd35
2019-09-13 18:38:22 -07:00
svcscm
c86178d881 Updating submodules
Summary:
GitHub commits:

147778cb75
f65c05bba6
2e8cae784a
9ba88a1e5d
f0dd9a4e88

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 38d79d7253a006cd05e4d024ac4444528e12d049
2019-09-13 11:18:46 -07:00
svcscm
9f7d85c637 Updating submodules
Summary:
GitHub commits:

8e45af9c4c
ed4b400015
5a4e5de517
544a63f02e
fddd67d14b
01c3f19961
3ffe790576
994d90e147

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: a104beddd09e8dfd7077d0fda4f5a440564ecf9a
2019-09-13 11:18:45 -07:00
svcscm
30aeddd3ca Updating submodules
Summary:
GitHub commits:

fbe5d64918
9431d78be4
0c5ceda68a
f58496300e
1118ea1999
b51cd36793
957df32bce
85fcaaa896
13b424652e

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 85d523f46124d20d710b8726b2248aea96d7c7fe
2019-09-12 23:57:47 -07:00
Hasnain Lakhani
267fddccc8 Add CMake option to fetch deps statically
Summary:
We would like to build a version of proxygen that has minimal
dependencies on dynamic libraries.

Reviewed By: yfeldblum

Differential Revision: D17228181

fbshipit-source-id: cfd61afdfa978c49a536184f426502196241fb8a
2019-09-12 23:35:45 -07:00
svcscm
a6cbd9ad2c Updating submodules
Summary:
GitHub commits:

c5b1078066
92a6ede4ef
b87f792ffd
a378a4c2ac
7f9dd385b0
b48dde82fa
ce183205ac
46b3738a1d
9ec189e499

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: f72bb0b0e577243b807c6e641629aa0311216174
2019-09-12 23:35:45 -07:00
svcscm
75381ad5b6 Updating submodules
Summary:
GitHub commits:

8d88865b18
3549556e95
88e03cf4ab
e58b72216e
ac9918c3e5
d35ffd569c
17b64199fd
68b0e4d85a
3d6c07527e
ac0ed8caa4

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 5bd72fbd7e1fc8fe0e0f69ab00aeb0d045348444
2019-09-12 14:16:24 -07:00
Adam Simpkins
c6de7c47ee always ensure that the contents of the .eden/ directory are correct
Summary:
Make sure the contents of the special `.eden/` subdirectory are correct each
time we mount a checkout.  Before we would generally only set up the contents
of this directory if it didn't previously exist.

Now the code verifies that the contents of each symlink in this directory are
correct, and recreates the symlink if needed.

This allows EdenFS to automatically repair the contents of this directory even
if the checkout or its `clients` directory has been manually moved.

Reviewed By: wez

Differential Revision: D17279413

fbshipit-source-id: e24e7530f44fff94ebb6f67174aaf78c9b498d6b
2019-09-11 22:02:16 -07:00
svcscm
faa6855117 Updating submodules
Summary:
GitHub commits:

e1edede646

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 82b30a2a997d68131df21bbdcfd0bc3aab924b29
2019-09-11 15:43:22 -07:00
Adam Simpkins
4e38a7c74f build edenfsctl with CMake
Summary:
Update the CMakeLists.txt to also build the Python-based `edenfsctl` command
line tool.

This requires switching most of the thrift rules to generate both C++ and
Python sources.

Note that one missing feature at this point is that this does not package
external dependencies into the binary.  Currently `edenfsctl` depends on both
`six` and `toml` as external dependencies.  For now these must be available in
your `PYTHONPATH` in order to run the generated `edenfsctl` binary.

Reviewed By: chadaustin

Differential Revision: D17127615

fbshipit-source-id: fc138ab39e75c6a5bbd39e3f527d4e9f7f420e46
2019-09-11 13:20:43 -07:00
Adam Simpkins
e7af87edad fix library dependencies on Windows
Summary:
The Windows library definitions were missing many dependencies.  They happened
to link for now, but this was fragile and it was easy to cause this to break
by making changes to the dependency structure of our other libraries.

Additionally the files in eden/fs/win/store and eden/fs/win/mount also have
circular dependencies between each other that cannot easily be resolved in one
direction or the other.  Therefore this combines the two `eden_win_mount` and
`eden_win_store` libraries into a single `eden_win` library to eliminate this
circular library dependency.

Reviewed By: wez

Differential Revision: D17309104

fbshipit-source-id: c95d0cfb8979e2b68d4947884acd13f859a4e01b
2019-09-11 13:20:43 -07:00
Adam Simpkins
69801222f7 split out thrift utilities from the eden_service library
Summary:
In the CMake-based build, split EdenError.cpp and PrettyPrinters.cpp out into
a separate library from `eden_service`.  Even though these are in the
`eden/fs/service/` subdirectory, these are lower-level utilities, and other
libraries in other `eden/fs/` subdirectories depend on them.  Having them be
part of the `eden_service` library incorrectly causes circular library
dependencies.  This splits the libraries out to make our CMake-based build
more closely match our Buck-based build.

Reviewed By: wez

Differential Revision: D17308275

fbshipit-source-id: 557af2ab54fb2a8baa44d30bcef85e4f76ab7cf0
2019-09-11 13:20:43 -07:00
svcscm
902f15e595 Updating submodules
Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 88ae4e26203b5cb9fa04d730c0fac57d8149cac6
2019-09-11 13:20:43 -07:00
svcscm
048512eecf Updating submodules
Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 0643fdce9eee21b5caea3af56f9638cf10cd1756
2019-09-11 12:24:04 -07:00
Adam Simpkins
a7f2d733a4 fbcode_builder: update FBPythonBinary.cmake to work on Windows
Summary:
On Windows we have to explicitly invoke `make_fbpy_archive.py` with python.
Therefore use CMake's built-in `FindPythonInterp` module to find the python
executable and use that when invoking `make_fbpy_archive.py`

This is slightly complicated by the effort required to find python with older
versions of CMake.  We ideally still want to support versions of CMake back to
at least 3.8, which means we need to still support finding Python with the
older `FindPythonInterp.cmake` module

Reviewed By: wez

Differential Revision: D17128606

fbshipit-source-id: 3f4beff76848b8a362ebdf21198e7a8bf1b0537f
2019-09-11 11:34:17 -07:00
Adam Simpkins
48682a439a add code to FileInode::stat() to handle unexpected tag values
Summary:
This silences a compiler warning about reaching the end of a non-void function
without returning a value.

Reviewed By: chadaustin

Differential Revision: D17308290

fbshipit-source-id: 95cdb3353364a36dcd2295b19bf745a941e5e3cf
2019-09-11 11:16:11 -07:00
svcscm
57ec0e1275 Updating submodules
Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 4753e836833f8065b64fa9febf870aecee5a25d7
2019-09-11 11:16:11 -07:00
svcscm
00bbcb09f2 Updating submodules
Reviewed By: 2d2d2d2d2d

fbshipit-source-id: f3664e2184148f2ddb04677b426d4daaaa6ea260
2019-09-10 14:50:25 -07:00
svcscm
57116a3d82 Updating submodules
Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 3cb3a9e6cef24fcd33b375b4c34f74d786f56d9c
2019-09-10 14:50:25 -07:00
svcscm
3d69d01c27 Updating submodules
Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 8dd2eee6db1878cc187173c641ac93b2c5daa2fe
2019-09-10 12:09:16 -07:00