5537225a02
Summary: In the initial stages of the windows port we had problems building rocksdb on windows, so we disabled it. These days we're able to build it and detect it--we even require it in the cmake code, but hadn't gotten around to telling the rest of the code that we can use it. This commit re-enables it in the build but leaves sqlite as the default engine until we're able to perform some benchmarking. Rocksdb itself has some build issues on Windows; it doesn't use cmake to locate dependencies, so even though we built snappy it doesn't know how to find it without modifying the source: https://github.com/facebook/rocksdb/blob/master/thirdparty.inc#L4 For that reason, we disable the use of Snappy in the Windows build. However, in the version of rocksdb that we were using, it would default to trying to use Snappy even though it wasn't compiled in and throw an exception. I've upgraded to a newer version of rocksdb that will simply not use compression if no compression was enabled at build time. Given that we mostly store relatively small objects, I'm assuming that the lack of compression is fine for now. Reviewed By: xavierd Differential Revision: D21319896 fbshipit-source-id: 2a2d06d4bd5382706e9220f9b4a2de99dc18311d |
||
|---|---|---|
| .. | ||
| CMake | ||
| getdeps | ||
| manifests | ||
| specs | ||
| .gitignore | ||
| docker_build_with_ccache.sh | ||
| docker_builder.py | ||
| docker_enable_ipv6.sh | ||
| fbcode_builder_config.py | ||
| fbcode_builder.py | ||
| getdeps.py | ||
| LICENSE | ||
| make_docker_context.py | ||
| parse_args.py | ||
| README.docker | ||
| README.md | ||
| shell_builder.py | ||
| shell_quoting.py | ||
| travis_docker_build.sh | ||
| utils.py | ||
Easy builds for Facebook projects
This is a Python 2.6+ library designed to simplify continuous-integration (and other builds) of Facebook projects.
For external Travis builds, the entry point is travis_docker_build.sh.
Using Docker to reproduce a CI build
If you are debugging or enhancing a CI build, you will want to do so from host or virtual machine that can run a reasonably modern version of Docker:
./make_docker_context.py --help # See available options for OS & compiler
# Tiny wrapper that starts a Travis-like build with compile caching:
os_image=ubuntu:18.04 \
gcc_version=7 \
make_parallelism=2 \
travis_cache_dir=~/travis_ccache \
./travis_docker_build.sh &> build_at_$(date +'%Y%m%d_%H%M%S').log
IMPORTANT: Read fbcode_builder/README.docker before diving in!
Setting travis_cache_dir turns on ccache,
saving a fresh copy of ccache.tgz after every build. This will invalidate
Docker's layer cache, foring it to rebuild starting right after OS package
setup, but the builds will be fast because all the compiles will be cached.
To iterate without invalidating the Docker layer cache, just cd /tmp/docker-context-* and interact with the Dockerfile normally. Note
that the docker-context-* dirs preserve a copy of ccache.tgz as they
first used it.
What to read next
The *.py files are fairly well-documented. You might want to peruse them in this order:
- shell_quoting.py
- fbcode_builder.py
- docker_builder.py
- make_docker_context.py
As far as runs on Travis go, the control flow is:
- .travis.yml calls
- travis_docker_build.sh calls
- docker_build_with_ccache.sh
This library also has an (unpublished) component targeting Facebook's internal continuous-integration platform using the same build-step DSL.
Contributing
Please follow the ambient style (or PEP-8), and keep the code Python 2.6
compatible -- since fbcode_builder's only dependency is Docker, we want to
allow building projects on even fairly ancient base systems. We also wish
to be compatible with Python 3, and would appreciate it if you kept that
in mind while making changes also.