a8b76ca8e3
Summary: In 0ae204a978c11ddefafd81bd319a078239a44c1c the 'projects_dir' option became a required constructor argument since it is called within the constructor. However, it has not been adjusted in the subclasses that used to set the option after instantiation. This commit fixes the 'shell_builder' and the 'debian_system_builder'. Pull Request resolved: https://github.com/facebook/openr/pull/50 Test Plan: 1. Go to build directory: `cd build` 2. Run the `shell_builder` & `debian_system_builder`: - `python fbcode_builder/shell_builder.py` - `python debian_system_builder/debian_system_builder.py` `shell_builder` output before: ``` Traceback (most recent call last): File "fbcode_builder/shell_builder.py", line 102, in <module> builder = ShellFBCodeBuilder() File "/home/butjar/tu/ma/openr/build/fbcode_builder/fbcode_builder.py", line 93, in __init__ self._github_dir = self.option('projects_dir') File "/home/butjar/tu/ma/openr/build/fbcode_builder/fbcode_builder.py", line 108, in option raise RuntimeError('Option {0} is required'.format(name)) RuntimeError: Option projects_dir is required ``` `shell_builder` output after: ``` set -exo pipefail export CCACHE_DIR='/home/butjar/.fbcode_builder-sZshomesZsbutjarsZstusZsmasZsopenr/.ccache' CC="ccache ${CC:-gcc}" CXX="ccache ${CXX:-g++}" ### Diagnostics ### # Builder ShellFBCodeBuilder(google/googletest:cmake_defines={u'BUILD_GTEST': u'ON', u'BUILD_SHARED_LIBS': u'OFF'}, google/googletest:git_hash=u'release-1.8.1', facebook/openr:local_repo_dir='/home/butjar/tu/ma/openr', facebook/zstd:git_hash=ShellQuoted(u'$(git describe --abbrev=0 --tags origin/master)'), openr/build:cmake_defines={u'ADD_ROOT_TESTS': u'OFF'}, thom311/libnl:git_hash=u'libnl3_2_25', projects_dir=u'/home/butjar/.fbcode_builder-sZshomesZsbutjarsZstusZsmasZsopenr', fmtlib/fmt:git_hash=u'5.3.0', wangle/wangle/build:cmake_defines={u'BUILD_TESTS': u'OFF'}, prefix=u'/home/butjar/.fbcode_builder-sZshomesZsbutjarsZstusZsmasZsopenr/installed', fizz/fizz/build:cmake_defines={u'BUILD_TESTS': u'ON'}, ccache_dir=u'/home/butjar/.fbcode_builder-sZshomesZsbutjarsZstusZsmasZsopenr/.ccache', zeromq/libzmq:git_hash=u'v4.2.2', make_parallelism=4, jedisct1/libsodium:git_hash=u'stable') hostname cat /etc/issue || echo no /etc/issue g++ --version || echo g++ not installed cmake --version || echo cmake not installed ### Check out fmtlib/fmt, workdir build ### mkdir -p '/home/butjar/.fbcode_builder-sZshomesZsbutjarsZstusZsmasZsopenr' && cd '/home/butjar/.fbcode_builder-sZshomesZsbutjarsZstusZsmasZsopenr' git clone https://github.com/'fmtlib/fmt' mkdir -p '/home/butjar/.fbcode_builder-sZshomesZsbutjarsZstusZsmasZsopenr'/'fmt'/'build' && cd '/home/butjar/.fbcode_builder-sZshomesZsbutjarsZstusZsmasZsopenr'/'fmt'/'build' git checkout '5.3.0' ### Build and install fmtlib/fmt ### ... ``` Reviewed By: steven1327 Differential Revision: D21865881 Pulled By: saifhhasan fbshipit-source-id: dfd78127d3b2c78721f84a3ecafe0b7198c38f06 |
||
---|---|---|
.. | ||
CMake | ||
getdeps | ||
manifests | ||
specs | ||
.gitignore | ||
docker_build_with_ccache.sh | ||
docker_builder.py | ||
docker_enable_ipv6.sh | ||
fbcode_builder_config.py | ||
fbcode_builder.py | ||
getdeps.py | ||
LICENSE | ||
make_docker_context.py | ||
parse_args.py | ||
README.docker | ||
README.md | ||
shell_builder.py | ||
shell_quoting.py | ||
travis_docker_build.sh | ||
utils.py |
Easy builds for Facebook projects
This is a Python 2.6+ library designed to simplify continuous-integration (and other builds) of Facebook projects.
For external Travis builds, the entry point is travis_docker_build.sh
.
Using Docker to reproduce a CI build
If you are debugging or enhancing a CI build, you will want to do so from host or virtual machine that can run a reasonably modern version of Docker:
./make_docker_context.py --help # See available options for OS & compiler
# Tiny wrapper that starts a Travis-like build with compile caching:
os_image=ubuntu:18.04 \
gcc_version=7 \
make_parallelism=2 \
travis_cache_dir=~/travis_ccache \
./travis_docker_build.sh &> build_at_$(date +'%Y%m%d_%H%M%S').log
IMPORTANT: Read fbcode_builder/README.docker
before diving in!
Setting travis_cache_dir
turns on ccache,
saving a fresh copy of ccache.tgz
after every build. This will invalidate
Docker's layer cache, foring it to rebuild starting right after OS package
setup, but the builds will be fast because all the compiles will be cached.
To iterate without invalidating the Docker layer cache, just cd /tmp/docker-context-*
and interact with the Dockerfile
normally. Note
that the docker-context-*
dirs preserve a copy of ccache.tgz
as they
first used it.
What to read next
The *.py files are fairly well-documented. You might want to peruse them in this order:
- shell_quoting.py
- fbcode_builder.py
- docker_builder.py
- make_docker_context.py
As far as runs on Travis go, the control flow is:
- .travis.yml calls
- travis_docker_build.sh calls
- docker_build_with_ccache.sh
This library also has an (unpublished) component targeting Facebook's internal continuous-integration platform using the same build-step DSL.
Contributing
Please follow the ambient style (or PEP-8), and keep the code Python 2.6
compatible -- since fbcode_builder
's only dependency is Docker, we want to
allow building projects on even fairly ancient base systems. We also wish
to be compatible with Python 3, and would appreciate it if you kept that
in mind while making changes also.