Summary:
Throws an exception when:
* The name specified in the manifest mismatches the filename
* Duplicated manifest -- with the sub-directory support it is now able to have multiple manifest files with the same name
Reviewed By: chadaustin
Differential Revision: D17438460
fbshipit-source-id: ac7ad0b701beb15f0e91bb05cd1ec8fe378ad5b6
Summary: Make getdeps to look for subdirectories for manifest files.
Reviewed By: simpkins
Differential Revision: D17222388
fbshipit-source-id: e13503beccd9edf6d80f78fbc3238b2a8d2053dd
Summary:
The libraries in thrift/lib/py support both Python 2 and Python 3, and rely on
the Python six module for some of this compatibility support.
Update the getdeps manifest for fbthrift to indicate this dependency, and
update fbthrift's CMakeLists.txt file to find and reference python-six
properly. This will ensure that the python-six code is built into any python
executable that uses the thrift/lib/py libraries.
Reviewed By: yfeldblum
Differential Revision: D17401218
fbshipit-source-id: 0007dda8974ae9bd87e4d7e256c74908c9a30d8f
Summary:
Add a new builder that can extract Python wheel files, and re-package them
for consumption by our add_fb_python_library() and add_fb_python_executable()
CMake functions. This is useful for dependencies on packages from PyPI.
At the moment this code only handles architecture-independent pure-Python
packages. It shouldn't be too hard to extend this to handle more complex
wheels, but for now I only need to use it for some pure-Python wheels and so I
haven't tested with more complex wheel files.
This also includes two new manifests for python-six and python-toml that take
use this new builder.
Reviewed By: wez
Differential Revision: D17401216
fbshipit-source-id: d6f74565887c3f004e1c06503dc9ec81599dd697
Summary:
Update the code to use `os.replace()` rather than `os.rename()` so that it
won't fail on Windows if the destination path already exists.
Reviewed By: chadaustin
Differential Revision: D17462716
fbshipit-source-id: cbc06319ccb2d73868f80ab1874890ebec5a621b
Summary: This enable test targets to be built and ran
Reviewed By: lnicco
Differential Revision: D17408942
fbshipit-source-id: 144d223bc3830d07a0420e9569d3166a8646cd9a
Summary: changes the way it pulls its dependencies
Reviewed By: wez
Differential Revision: D17363235
fbshipit-source-id: 37e509c7e413f763e3553e9f01ac23951045089c
Summary:
Update the thrift C++ and Python CMake rules to indicate that the output also
depends on the thrift compiler itself.
Previously the C++ rule indicated that the output depended on the thrift
template files, which caught most cases when the thrift compiler was updated,
but wasn't fully correct. The thrift templates were also removed and baked
into the thrift compiler binary in D16356056.
Reviewed By: yfeldblum, chadaustin
Differential Revision: D17401217
fbshipit-source-id: ae5cde7a7e5e07a74406a1b6f4469124187bc12f
Summary:
This is useful when working on changes to some of the builder functions,
to skip ever trying to use cached results.
Reviewed By: chadaustin
Differential Revision: D17401219
fbshipit-source-id: fb7d5ea45618653957b9bd6a62eed91d8334824d
Summary:
This commit teaches fixup-dyn-deps how to generate correct
absolute paths in the context of the ultimate install path (specified
via the `--final-install-prefix` option)
Absolute paths are desirable if you have, for example, an executable
that you wish to install with the setuid bit set.
Reviewed By: simpkins
Differential Revision: D17361491
fbshipit-source-id: 4c4f3f15208266300622f84dc9cd1ba83883dfb7
Summary:
Add a license header to satisfy the open source linter. Use the same
header the other .cmake files have.
Reviewed By: mhlakhani
Differential Revision: D17404782
fbshipit-source-id: 66679d72c9e680f8bb8b27869e981a046b3520cf
Summary:
This commit adds a getdeps command that is able to generate
a workflow file for the GitHub Actions CI environment.
The workflow file could be expressed more simply using the matrix
syntax and with three steps (checkout, build, test), but I chose to
break out the steps for each of the dependencies because the UX
while waiting on the build is much nicer that way: the steps show
during and live log tailing for the section of the build that is
underway. If they were all lumped into a single build step then
the logs from the boost section of the build dominate and make
the github UI work very hard.
Pull Request resolved: https://github.com/facebook/watchman/pull/743
Test Plan:
https://github.com/facebook/watchman/pull/743 successfully
executes the github actions CI flow.
```
$ opensource/fbcode_builder/getdeps.py generate-github-actions --output-file watchman/.github/workflows/main.yml watchman
```
Reviewed By: simpkins
Differential Revision: D17384915
Pulled By: wez
fbshipit-source-id: 9a9e5a3e806c18f6cc38ba1cb7059740cda01ad4
Summary:
GitHub Actions CI `windows-latest` environment has only VS 2019
installed, so we need to expand our logic to be able to locate it.
Note that Boost 1.69 doesn't know how to locate VS 2019 so we are effectively
tied to VS 2017 at the moment; the search order in this diff reflects that.
(That means that we can't target `windows-latest` on GitHub Actions, but that
is really a concern for a later diff in this stack)
Reviewed By: simpkins
Differential Revision: D17385052
fbshipit-source-id: 9bb0612154f42d425a625406488f39bb4ec3d8ae
Summary:
while testing https://github.com/facebook/watchman/pull/743 I
noticed that the cmake builds were picking up the installed mingw GCC
compiler rather than the MSVC compiler. That would be fine except that
boost is built with MSVC and its generated libraries cannot be subsequently
found by a cmake gcc build that uses FindBoost.
This commit forces cmake to pick cl.exe rather than gcc. This is probably
fine to do unconditionally on windows, but since I've only observed this
particular problem with GitHub Actions I'm keeping it constrained to that
environment for now.
Reviewed By: simpkins
Differential Revision: D17385050
fbshipit-source-id: 90bef898b138e5d4bbd28a155ed1bd468acee9de
Summary:
We've been squeaking by with assuming that flex is installed already
on posix systems, but that isn't the case on the github actions default
configuration.
Adjust the bison recipe: on windows it deploys both flex and bison. We use the
same source for both flex and bison but install flex to a separate install
prefix to make it easier to consume the flex dependency distinct from the bison
dependency.
The latest flex release segfaults during compilation on linux unless we
force -DGNU_SOURCE, so the manifest does that on linux.
Reviewed By: simpkins
Differential Revision: D17385051
fbshipit-source-id: 9f31b07849af9de50099d1b20bedba517bbbdf2f
Summary:
while testing https://github.com/facebook/watchman/pull/743 on macOS
I noticed that the libevent build failed to find openssl.
openssl is special on macos because apple do not ship the headers.
We already build and depend on openssl for the folly manifest on
macos, so this is really just adding a missing edge.
Reviewed By: simpkins
Differential Revision: D17385053
fbshipit-source-id: 1b688537fef422d81a959fc5749c871b9e868baa
Summary:
We would like to build a version of proxygen that has minimal
dependencies on dynamic libraries.
Reviewed By: yfeldblum
Differential Revision: D17228181
fbshipit-source-id: cfd61afdfa978c49a536184f426502196241fb8a
Summary:
On Windows we have to explicitly invoke `make_fbpy_archive.py` with python.
Therefore use CMake's built-in `FindPythonInterp` module to find the python
executable and use that when invoking `make_fbpy_archive.py`
This is slightly complicated by the effort required to find python with older
versions of CMake. We ideally still want to support versions of CMake back to
at least 3.8, which means we need to still support finding Python with the
older `FindPythonInterp.cmake` module
Reviewed By: wez
Differential Revision: D17128606
fbshipit-source-id: 3f4beff76848b8a362ebdf21198e7a8bf1b0537f
Summary: [Thrift] Bake templates into the compiler binary rather than having to package the templates with the compiler.
Reviewed By: stevegury, vitaut
Differential Revision: D16356056
fbshipit-source-id: b971dce7aada8dc2a26587fbf818e661acd0c555
Summary:
This should make it easier to eg: enable fbzmq on macos at a later
time, and also makes things more explicit about what is being built, and
importantly, by listing these in the manifest ensures that we have a
hash change if we change this list; we wouldn't trigger such a change
if the list were encoded solely in builder.py
Reviewed By: chadaustin
Differential Revision: D17133149
fbshipit-source-id: caf0dd45e262188eeefafe0868ef95ad257a4950
Summary:
* Fix a minor issue with `add_fb_python_executable` where it would
emit `--main` even if `MAIN_MODULE` had not been specified
* Introduce the ability to rename python sources in the manifest so
that they have a different install path.
* Note that this changes the DEPENDS parameters to reference abs_sources;
previously these were using relative paths to the sources pulled directly
from the SOURCES arguments, but since those can now contain aliasing
operations we need to pre-process the list. I switched to abs_sources
because that was already being maintained. If for some reason the
DEPENDS list needs to be relative paths, this will need to be adjusted.
Reviewed By: simpkins
Differential Revision: D17147417
fbshipit-source-id: d0825bfbdad6be658c078aaac6645e6926a1530f
Summary:
This diff adds a small abstraction that allows for uploading
and downloading from an artifact cache.
We try to download from this cache at build time, but will only
try to populate it for continuous builds--those are built from
code that has been reviewed and landed on master. This restriction
helps to avoid thrashing the cache with works in progress and
results in a slightly more trustworthy state of the cache contents.
In addition to this, we choose only to cache third party projects.
The rationale is that our first party projects move too quickly to
be worth caching, especially since the cache granularity is for
the whole project rather than just changed elements of a given
project.
In a later diff I will introduce some implementations of the
cache class that work with eg: Travis or Circle CI caching.
Reviewed By: simpkins
Differential Revision: D16873307
fbshipit-source-id: 2bfb69e36615791747b499073586562f2ca48be9
Summary: [OpenSource] Cut `mstch` manifest, which was but is no longer used by `fbthrift`.
Reviewed By: avalonalex, davidtgoldblatt
Differential Revision: D17160816
fbshipit-source-id: 3b96142455d6990870e8ed3e5ce330d339d5a657
Summary:
Feed the first-party oncall data through to the testpilot
invocation. This will set the owner of the test in continuous runs.
The oncall is passed through via the sandcastle module, which means
that you will have to manually pass it through for local testing
(see test plan), but it should automatically get picked up when
sandcastle schedules the `test` step of the job.
Reviewed By: chadaustin
Differential Revision: D17146802
fbshipit-source-id: a4f0e65853a46ed8709594c96db859ede2530b00
Summary: when trying to build openr with `getDeps.py` constant definitions where not available since they were not part of the generated thrift lib. this fixes it.
Reviewed By: chadaustin
Differential Revision: D16064417
fbshipit-source-id: ee2b3e061caf67235f4a873405cf6cd6fb663ef4
