sapling/build/fbcode_builder
Adam Simpkins 2e09bbac2a fbcode_builder: cmake: propagate thrift include dependencies correctly
Summary:
Update `add_thrift_cpp2_library()` to pass in the correct `-I` flags when
invoking the thrift compiler so that it can find all of the other thrift files
that this library depends on.

D16062657 was a previous attempt to do this, but suffered from a few problems:
- It required all dependencies to be defined before
  `add_thrift_cpp2_library()` was called.  This requires users to carefully
  order their CMake files and subdirectory include ordering.
- It only handled one level of dependencies, and did not propagate include
  paths for deeper dependencies.
- It set the include path for dependencies to the source directory path where
  the dependency was originally built, rather than the directory where the
  thrift file for that dependency would be installed.

This change does require CMake 3.8+.  Previous versions of CMake do not
support using generator expressions to generate multiple arguments for a
custom command.

Reviewed By: strager

Differential Revision: D17005381

fbshipit-source-id: 31190beba94b4d1010445375a5e2791450230f7d
2019-08-28 11:58:01 -07:00
..
CMake fbcode_builder: cmake: propagate thrift include dependencies correctly 2019-08-28 11:58:01 -07:00
getdeps getdeps: fixup testpilot new test recording 2019-08-20 15:43:32 -07:00
manifests break oss build dependency on fboss/common/ 2019-08-27 17:15:54 -07:00
specs fbcode_builder: update the old fbcode_builder spec for fbzmq 2019-08-26 11:47:03 -07:00
.gitignore add fbcode_builder sources 2019-04-26 11:31:58 -07:00
docker_build_with_ccache.sh add fbcode_builder sources 2019-04-26 11:31:58 -07:00
docker_builder.py add fbcode_builder sources 2019-04-26 11:31:58 -07:00
docker_enable_ipv6.sh add fbcode_builder sources 2019-04-26 11:31:58 -07:00
fbcode_builder_config.py add fbcode_builder sources 2019-04-26 11:31:58 -07:00
fbcode_builder.py Fix gmock handling in rsocket opensource build 2019-07-30 23:00:11 -07:00
getdeps.py getdeps: allow overriding project source, build, and install directories 2019-08-15 17:58:29 -07:00
make_docker_context.py update docker os_image to ubuntu18 and gcc7 2019-05-10 16:39:16 -07:00
parse_args.py add fbcode_builder sources 2019-04-26 11:31:58 -07:00
README.docker add fbcode_builder sources 2019-04-26 11:31:58 -07:00
README.md update README.md on ubuntu and gcc version 2019-05-27 13:33:44 -07:00
shell_builder.py add fbcode_builder sources 2019-04-26 11:31:58 -07:00
shell_quoting.py add fbcode_builder sources 2019-04-26 11:31:58 -07:00
travis_docker_build.sh add fbcode_builder sources 2019-04-26 11:31:58 -07:00
utils.py add fbcode_builder sources 2019-04-26 11:31:58 -07:00

Easy builds for Facebook projects

This is a Python 2.6+ library designed to simplify continuous-integration (and other builds) of Facebook projects.

For external Travis builds, the entry point is travis_docker_build.sh.

Using Docker to reproduce a CI build

If you are debugging or enhancing a CI build, you will want to do so from host or virtual machine that can run a reasonably modern version of Docker:

./make_docker_context.py --help  # See available options for OS & compiler
# Tiny wrapper that starts a Travis-like build with compile caching:
os_image=ubuntu:18.04 \
  gcc_version=7 \
  make_parallelism=2 \
  travis_cache_dir=~/travis_ccache \
    ./travis_docker_build.sh &> build_at_$(date +'%Y%m%d_%H%M%S').log

IMPORTANT: Read fbcode_builder/README.docker before diving in!

Setting travis_cache_dir turns on ccache, saving a fresh copy of ccache.tgz after every build. This will invalidate Docker's layer cache, foring it to rebuild starting right after OS package setup, but the builds will be fast because all the compiles will be cached. To iterate without invalidating the Docker layer cache, just cd /tmp/docker-context-* and interact with the Dockerfile normally. Note that the docker-context-* dirs preserve a copy of ccache.tgz as they first used it.

What to read next

The *.py files are fairly well-documented. You might want to peruse them in this order:

  • shell_quoting.py
  • fbcode_builder.py
  • docker_builder.py
  • make_docker_context.py

As far as runs on Travis go, the control flow is:

  • .travis.yml calls
  • travis_docker_build.sh calls
  • docker_build_with_ccache.sh

This library also has an (unpublished) component targeting Facebook's internal continuous-integration platform using the same build-step DSL.

Contributing

Please follow the ambient style (or PEP-8), and keep the code Python 2.6 compatible -- since fbcode_builder's only dependency is Docker, we want to allow building projects on even fairly ancient base systems. We also wish to be compatible with Python 3, and would appreciate it if you kept that in mind while making changes also.