Summary:
boost.fiber is not available on macos and is required by fbzmq,
so disable the build on darwin.
In D17128935 I tried to make libzmq build with cmake to build
on windows, but there's a problem with its cmake and it fails
to configure.
So let's just nop out the build for !linux for now.
Reviewed By: jstrizich
Differential Revision: D17129362
fbshipit-source-id: db89010d18707a4105b8d395b1758f738336495d
Summary:
This module provides a `add_fbthrift_library()` convenience function that can
be used to generate thrift libraries for multiple languages with a single
call.
Reviewed By: wez
Differential Revision: D16742275
fbshipit-source-id: c13120cf9e051629951282141eb22f3e085782f9
Summary:
This module helps generate Python libraries from thrift files.
It is the Python equivalent of `FBThriftCppLibrary.cmake`
Reviewed By: wez
Differential Revision: D16742274
fbshipit-source-id: ef0dc7f24eb1c7375d2587ee6b71fd25bd6f35a7
Summary:
Fix a minor bug in the argument parsing for `add_fbthrift_cpp_library()`:
only a single value should be supplied for the `THRIFT_INCLUDE_DIR` argument.
This parameter should not accept a list of arguments.
Reviewed By: strager
Differential Revision: D17125810
fbshipit-source-id: 31f354e4d6299d3626a94f3b4b5ff8c58bb0cf7b
Summary:
Rename the `ThriftCppLibrary.cmake` file to `FBThriftCppLibrary.cmake`, and
also rename `add_thrift_cpp2_library()` to `add_fbthrift_cpp_library()`.
Explicitly calling this `fbthrift` helps clearly distinguish that this is
intended for use with fbthrift (https://github.com/facebook/fbthrift/), as
opposed to Apache thrift.
Reviewed By: wez
Differential Revision: D16738440
fbshipit-source-id: 9b255e06b71c98ad74a34989f564a211958dcdd5
Summary:
Update install_fb_python_library() to work with CMake versions older than 3.7.
Previously the code used a generator expression in the directory argument
supplied to `install(DIRECTORY)`. Support for generators in this parameter
was only added in CMake 3.5, but it was buggy and did not correctly handle
absolute paths until 3.7.
This updates the code to avoid using a generator expression. This does
require that the corresponding `add_fb_python_library()` statement appear
before the `install_fb_python_library()` statement, but this seems like a
pretty reasonable restriction.
Reviewed By: chadaustin
Differential Revision: D17103597
fbshipit-source-id: 54d6af93a3f530373cfe3137db99436d8484f308
Summary:
Update the generated `run_cmake.py` script to use `os.execve()` rather than
`subprocess.call()`, so that it now propagates CMake's return code back to its
caller.
Reviewed By: chadaustin
Differential Revision: D17089206
fbshipit-source-id: e01f05f492ccb842d4967e59fd0bc9a3e59b8a42
Summary:
Fix printing the manifest context in the error message if we cannot find a
project fetcher. Previously the context in the message would be printed as
something like `<getdeps.manifest.ManifestContext object at 0x7fcce987e610>`,
now it shows instead as something like
`{distro=ubuntu, distro_vers=18.04, fb=off, os=linux, test=off}`
Reviewed By: chadaustin
Differential Revision: D17089208
fbshipit-source-id: c16549b61030d813b7b5ff9f65966436dc1e1898
Summary:
This just strips off a trailing '/' from the `INSTALL_DIR` property that we
set on python targets, to avoid having double-slashes in the paths that we use
during installation. This shouldn't really have any material difference other
than cleaning up the paths that get printed during the installation phase.
Reviewed By: chadaustin
Differential Revision: D17089207
fbshipit-source-id: ab36bb76c19fa60fe037f7a5290ccfd6bdbf13b0
Summary:
Update fboss to use `ThriftCppLibrary.cmake` directly from fbcode_builder
rather than maintaining its own copy of this file.
Reviewed By: chadaustin
Differential Revision: D17005421
fbshipit-source-id: a64df426118df6088b47f09410dad7b8b7e79a43
Summary:
Update `add_thrift_cpp2_library()` to pass in the correct `-I` flags when
invoking the thrift compiler so that it can find all of the other thrift files
that this library depends on.
D16062657 was a previous attempt to do this, but suffered from a few problems:
- It required all dependencies to be defined before
`add_thrift_cpp2_library()` was called. This requires users to carefully
order their CMake files and subdirectory include ordering.
- It only handled one level of dependencies, and did not propagate include
paths for deeper dependencies.
- It set the include path for dependencies to the source directory path where
the dependency was originally built, rather than the directory where the
thrift file for that dependency would be installed.
This change does require CMake 3.8+. Previous versions of CMake do not
support using generator expressions to generate multiple arguments for a
custom command.
Reviewed By: strager
Differential Revision: D17005381
fbshipit-source-id: 31190beba94b4d1010445375a5e2791450230f7d
Summary:
All of the requisite headers are now open sourced and eden
has been converted to use them. This removes any logic that references
this directory and kills the now unused cmake file.
Reviewed By: chadaustin
Differential Revision: D16994732
fbshipit-source-id: 573b8b50540ee64590682be2a54aa94659f12368
Summary:
Most stat code in fbcode was moved in to open source under
fbcode/fb303. The only exception that we use internally is the
MonotonicCounter class. This diff moves all of our stats code to use
the open source headers and updates cmake files to pull in the open
source fb303.
Reviewed By: chadaustin, shri-khare
Differential Revision: D16969960
fbshipit-source-id: 12b7abb54c956c242c8e27eb69fd96925c7e61f7
Summary:
Update add_thrift_cpp2_library() to add a PUBLIC_HEADER property to the
generated library target, so that the generated headers can be installed using
an `install()` call. Also add a `HEADER_INSTALL_DIR` property which indicates
the directory they should be installed into. A `INCLUDE_DIR` argument was
added to customize the include installation path, should anyone care to do so.
This also removes code that was previously incorrectly installing the
generated headers into the source tree.
Reviewed By: wez
Differential Revision: D16853404
fbshipit-source-id: 6f640b2bd347c99804e074fca0209dda37c8a9cf
Summary:
The external Travis CI builds for fbzmq are still using the older
fbcode_builder spec (as opposed to the newer getdeps manifest file).
D16577367 moved the CMakeLists.txt file to the top-level directory, but the
fbzmq spec file was still looking for it in the fbzmq subdirectory.
Reviewed By: jstrizich
Differential Revision: D17005361
fbshipit-source-id: 3f7664eadfb60ec7606124a14445b44ae586b8a7
Summary:
Fix the fbzmq pathmap listed in its manifest file to match the path map
actually used by ShipIt. The fact that this was broken was why internal
getdeps builds did not detect that D16577367 broke the build.
Reviewed By: wez, jstrizich
Differential Revision: D17005360
fbshipit-source-id: 046ff58ad70c03b860c3fccebaba975808df244d
Summary:
Update ThriftCppLibrary.cmake to record that the output depends on the thrift
`cpp2` template files. The `THRIFT_TEMPLATE_FILES` variable was defined in
D16062657 but doesn't appear to be used anywhere. This uses it, and also
updates it to only match the C++ template files.
Reviewed By: wez
Differential Revision: D16738442
fbshipit-source-id: aa4097e9ec1d009ef1e3ee2efd03ffa19a82ed12
Summary:
Add explicit double quotes around arguments that are intended to be exactly a
single parameter. Also line wrap to 80 characters.
Reviewed By: wez
Differential Revision: D16738443
fbshipit-source-id: b8c9b6d50da72b44e23eaf5effbc384b6cbbc1c0
Summary:
Change ThriftCppLibrary to use `fb_cmake_parse_args()`
This code doesn't really care much about handling empty arguments, so we could
use the standard `cmake_parse_arguments()` function here instead of
`fb_cmake_parse_args()`. However, it is slightly more convenient that
`fb_cmake_parse_args()` automatically errors out on unhandled arguments, so
that we don't have to check for this ourselves.
Reviewed By: wez
Differential Revision: D16738445
fbshipit-source-id: 67b9761f2ceaf7665bd336290bb44af32fec6f4a
Summary:
The manifest file for iproute2 previously listed the commit to check out as
7ca63aef7d1b0c808da0040c6b366ef7a61f38c1. However, this is not a commit, but
a tag object that refers to commit 92a0236a3cdf3438000834121b7ea8a09f1f52b1.
This caused getdeps to think that iproute2 was always out-of-date, even when
it was already on the correct commit, since it was comparing current commit ID
against the tag ID.
This updates the manifest to refer to the commit ID instead of the tag ID.
Reviewed By: shri-khare
Differential Revision: D17005418
fbshipit-source-id: c7145c73b66ae93a2e9097ddcf8c53237b135d32
Summary:
This simplifies things for the OSS CI situation in
the short term, especially since Eden isn't quite buildable
or usable outside FB at this time.
Reviewed By: chadaustin
Differential Revision: D16953346
fbshipit-source-id: 028150bb3cfc9a8ccbe1197321214b56ad359463
Summary:
does not affect the github build.
this is for fb CI
Reviewed By: dddmello
Differential Revision: D16935982
fbshipit-source-id: 61656fa600f19438b5aa89486486ed4ee103dea4
Summary:
In the Facebook infrastructure the testpilot runner talks
to a service to determine whether a given test should be run locally.
The remote service was returning an error for some tests because they
didn't have enough configuration specified.
This diff associates some configuration with the tests based on the
host type and makes things happier.
Reviewed By: Ben0mega
Differential Revision: D16894682
fbshipit-source-id: b6f59b112991160ef26cc126e98041d8908a6273
Summary:
Add some CMake functions for building standalone executables from Python
source files. This generates executables similar to PEX
(https://github.com/pantsbuild/pex).
In the future this could potentially be leveraged to directly build XAR files
(https://github.com/facebookincubator/xar).
The main advantages of these functions is that they allow easily defining
"libraries" of python files, and their dependencies, which can then be used
and packaged as part of multiple different standalone executables.
Reviewed By: wez
Differential Revision: D16722499
fbshipit-source-id: e1d829b911dc428e5438b5cf9cebf99b3fb6ce24
Summary:
Update the generated `run_cmake.py` script to allow the caller to specify that
they just want to run a build without the install step.
Reviewed By: wez
Differential Revision: D16778007
fbshipit-source-id: 1859aca2b80fa7b099b4790682a6508e0185f2a0
Summary:
This cleans up how the `CMAKE_ENV` and `CMAKE_DEFINE_ARGS` variables are
written in the generated `run_cmake.py` script that we emit for CMake-based
projects.
We now emit each entry in these variables on separate lines, just to improve
readability. (Both of these variables tend to have a number of entries and
are very long if emitted on a single line.)
This also replaces the `-DCMAKE_INSTALL_PREFIX` entry in `CMAKE_DEFINE_ARGS`
to have it correctly honor the `INSTALL_DIR` variable defined in
`run_cmake.py`. This makes `run_cmake.py` still do the right thing if someone
manually edits it to change the `INSTALL_DIR` value.
Reviewed By: wez
Differential Revision: D16778006
fbshipit-source-id: fee5d25748b87b5d9c57ee2edf8de5e586e872ee
Summary:
Add arguments to getdeps.py to allow overriding the source, build, and install
directories a per-project basis. The arguments take the form `[PROJECT:]PATH`
If the `PROJECT` portion is omitted, it defaults to the current project being
built.
In particular this makes it possible to specify `--src-dir .` to tell
getdeps.py to find the project sources from the current directory rather than
downloading them.
Reviewed By: wez
Differential Revision: D16778011
fbshipit-source-id: f33b87213ace04abb66334f588babdf59df91964
Summary:
This makes getdeps.py no longer crash if used in the folly repository.
Folly does not depend on any other Facebook projects, so it does not
include a `build/deps` directory.
Reviewed By: wez
Differential Revision: D16778009
fbshipit-source-id: 41be53d862f41b62154b081eb90ddba8742658fe
Summary:
Most of the getdeps subcommands operate on a single project, and some of the
argument parsing and initial logic to load the project is largely the same.
This consolidates that logic into a base class so that we can share this code
across subcommands, instead of repeating it.
This also unifies the behavior so that by default all commands enable tests on
the specified project, and disable test on dependent projects. Making sure
all commands use the same behavior here is important as whether are not tests
are enabled can affect the project configuration, and therefore affect its
getdeps hash.
Reviewed By: wez
Differential Revision: D16778010
fbshipit-source-id: 044f99ad6cdd4a56f843276cec8ead786249ee7a
Summary:
Fix the BuildOptions class to correctly honor the `host_type` parameter that
it was constructed with when constructing the manifest evaluation context.
I accidentally broke this behavior in D16477396, and incorrectly had this code
path default to using the current host system rather than the value passed in
from the command line.
Reviewed By: wez
Differential Revision: D16779579
fbshipit-source-id: de911daaa643f6303fd35149775ab25d3f64d34f
Summary:
In response to review feedback for D16477400 and D16477401, update
`ManifestLoader.load_all_manifests()` to only update its data for projects
that have not previously been loaded. This helps ensure that code using a
single `ManifestLoader` object cannot have two in-memory `Manifest` objects
for the same project, and that existing data (such as project hashes) can't be
invalidated if a manifest is later loaded from updated on-disk data.
Reviewed By: pkaush
Differential Revision: D16586682
fbshipit-source-id: 50b1979ec55f2ad6901629cd852293a8f6ca903f
Summary:
While developing on a project it is often convenient to be able to invoke its
build manually, rather than always needing to re-run `getdeps.py`. This
updates the CMakeBuilder to also emit a script that can be used to manually
run CMake outside of `getdeps.py`.
The CMakeBuilder is the only builder that this really matters for right now,
as pretty much all of the projects where we do first-party development use
CMake for their build system.
Reviewed By: pkaush
Differential Revision: D16477399
fbshipit-source-id: c8a14af158af7b32d6c799ef685b037e68b748ff
Summary:
Move code that computes project hashes to ManifestLoader. ManifestLoader is
the only class that has all of the information necessary to compute the
project hashes correctly. The ManifestLoader object can also cache previously
computed hashes, so that we don't have to keep computing hashes for projects
over and over again. Previously the `BuildOptions.compute_dirs()` function
would end up re-computing hashes for all dependencies each time it was called.
Reviewed By: strager
Differential Revision: D16477401
fbshipit-source-id: ce03642114f91ce4f859f612e6b2e747cf1653be
Summary:
The ManifestLoader contains all of the state needed to create a fetcher
object, so define a helper method on this object to create a fetcher.
Reviewed By: strager
Differential Revision: D16477395
fbshipit-source-id: 6de0942fe6b8de26c18c82bf99343f5467dc006a
Summary:
Add a new ManifestLoader class to handle loading manifests and computing
dependencies.
For now the main thing this class does is maintain the `manifest_by_name`
mapping. In subsequent diffs we should be able to move some additional logic
into this class, which will help clean up the code and eliminate some redudant
work. In particular, we can have this class cache project hashes, which will
avoid re-computing hashes over and over again for the same projects as we do
in many cases today. We should also be able to save and re-use some of the
project dependency ordering computation in some cases as well.
Reviewed By: strager
Differential Revision: D16477400
fbshipit-source-id: f06f62f77d8443fccaa69fe4c1306e39c395b325
Summary:
Update `BuildOpts.compute_dirs()` to use the correct project-specific manifest
context when computing project hashes. Previously it was incorrectly using
the initial project's context when evaluating all dependencies. This would
result in some projects potentially seeing the wrong values for variables that
may change from project to project (like `test`).
Reviewed By: pkaush
Differential Revision: D16477398
fbshipit-source-id: 6c23f5e5e19b2402000a138b3920b79044446041
Summary:
Add a ContextGenerator class so that we actually use the correct per-project
context when loading projects and computing dependencies.
Previously commands like `build` and `test` would change the contexts for each
project as they iterated through and performed the build. However, they did
not do this when first loading the projects. This could cause them to use
different context values when loading dependencies than when performing the
build. For instance, this could cause issues if a project depends on
`googletest` only when testing is enabled, as the code previously did not set
the "test" parameter when evaluating dependencies.
Reviewed By: pkaush
Differential Revision: D16477396
fbshipit-source-id: c1e055f07de1cb960861d19594e3bda20a2ccd87
Summary:
Check that all variable names are valid when loading manifest files.
This ensures that getdeps.py will error out if someone makes a typo in a
variable name, rather than treating it as never equal to anything.
Reviewed By: pkaush
Differential Revision: D16477397
fbshipit-source-id: 030e0642ff4a08db8eb74a0a0223e03d53e4880f
Summary: bring this in line with other facebook opensource projects
Reviewed By: saifhhasan
Differential Revision: D16577367
fbshipit-source-id: d762658505f824cc180c55ea4485cecf525b8fdc
Summary:
Simply linking `GMOCK_LIBS` into a binary was not telling CMake that the binary depends on `gmock` being built. So, let's add that dependency explicitly.
This wasn't breaking in production because we typically build with `-j 4`, and `gmock` was getting built before the first dependent binary would attempt to link.
Also, since `rsocket` bundles its own `gmock`, it is just a waste of time to compile a system-level gmock. It's not a real dependency.
NB: The change in `fbcode_builder.py` is needed because now that `rsocket` no longer depends on anything on Github, driver programs that were unconditionally setting `projects_dir` started to fail to build `rsocket`.
Reviewed By: simpkins
Differential Revision: D16461572
fbshipit-source-id: 1e95654e96256e7ed37d42e702b5433bf2fe5328
Summary:
flake8 (in Python 2.7 mode) complains that `typing` is mentioned in type annotations but is not defined:
```
fbcode_builder/getdeps/buildopts.py:251:21: F821 undefined name 'typing'
subst_mapping, # type: typing.Mapping[str, str]
^
fbcode_builder/getdeps/buildopts.py:253:5: F821 undefined name 'typing'
# type: (...) -> typing.Optional[str]
^
2 F821 undefined name 'typing'
2
```
Import `typing` explicitly to silence this warning.
Because `typing` may be unavailable, import it conditionally. (Because it's only referenced in comments, failing to import `typing` should have no effect at run time.)
Reviewed By: snarkmaster
Differential Revision: D16435696
fbshipit-source-id: 78a4a7b07acc46aa998f02b54b1a6e52c1daafde
Summary: Add a dependency from the eden open source build to the fb303 open source build and switch EdenServiceHandler to BaseService.
Reviewed By: simpkins
Differential Revision: D15528156
fbshipit-source-id: 2ca5c31dd9fcc9bac43fd399b27f33b6f2c5ebfc
Summary:
Some people want to use getdeps with Python 2.7. This looks easy to do, so take a step toward Python 2.7 support by fixing getdeps' tests when run with Python 2.7.
For Python 3, this diff should not change behavior.
This diff should address https://github.com/facebook/bistro/issues/35.
Reviewed By: snarkmaster
Differential Revision: D16435667
fbshipit-source-id: f5c262b12995b609263341c4de26dac7f9b12b70
Summary:
getdeps allowed common arguments to be specified either before or after
the subcommand name. However, the arguments were only actually honored
if they were specified on the command line after the subcommand name.
Specifying any of these arguments before the subcommand would simply be
ignored, and the default value would be used. This happened since the
subcommand argument's default value overwrote the value specified by the
user from the main command parser.
This fixes the behavior by using `argparse.SUPPRESS` so that argparse
won't set the value at all if it was not specified. This allows the
value from the main parser to be used correctly if the argument was not
seen by the subcommand parser.
Reviewed By: chadaustin
Differential Revision: D16435358
fbshipit-source-id: f3893ef7a27fb018247f249f313d6438472e0140
Summary:
If `path_search()` returns that CMake is not available, raise an Exception and
fail the build. This makes the failure somewhat easier to to identify.
Without this the code would continue and would try to invoke `subprocess` with
a value of `None` in the command argument list. This error is slightly harder
to debug, since it isn't clear which command argument or environment variable
is not a string.
Reviewed By: chadaustin
Differential Revision: D16354623
fbshipit-source-id: be972b02cb13bc70db0f867da70e5bf4c6cec46d
Summary:
The scratch path is used as part of the hash computation for each project. We
need to make sure this path is always normalized to ensure that we compute the
hashes consistently.
Reviewed By: chadaustin
Differential Revision: D16354624
fbshipit-source-id: 39b5362620bdc247cd7e7f1333dac319b354dc6f
Summary:
getdeps currently ends up calling `path_search()` repeatedly searching for
various C/C++ compilers in $PATH. It ends up doing this multiple times for
each dependency as it computes the project hashes. This updates the
`path_search()` function to cache its results so that we don't keep performing
the same searches on the file system over and over again.
Reviewed By: chadaustin
Differential Revision: D16354625
fbshipit-source-id: 116293bd2f636632517d26436b2332e6c10624f1
