sapling/build/fbcode_builder/manifests
Wez Furlong bd2688aae5 getdeps: allow satisfying deps from system packages
Summary:
From the outset, we wanted to be sure that getdeps was able
to source and build the dependencies so that we knew that we'd have
a repeatable build.  This came at the cost of build times: having
to build boost on each CI run is a bit of a chore.

This commit adds three new elements to the manifest files:

* `rpms` - a list of RPM names that are all required to be present
  in order to consider the dependency satisfied
* `debs` - like `rpms` above, but scoped to debian package names
* `preinstalled.env` - a list of environment variables that if they
  are all set and non-empty will satisfy the dependency.

A new `--allow-system-packages` option to getdeps enables the new
logic that looks for system packages; it is off by default, but
enabled in the generated GitHub Actions workflows.

A new `install-system-deps` subcommand is provided that will attempt
to install the system packages needed to satisfy the build.  This
typically needs to be run via sudo and is thus broken out separately
from the main getdeps build flow.

I made a pass over the manifest files and added package names that
satisfy the build on ubuntu-18 and fedora-31.

shri-khare: I renamed the `Python3.7.6` manifest to just `python` as
part of this change; the version of python that it pulls in through
the normal build is the same and I believe that an equal or newer
version of python3 is available in the GH actions builder.

The `preinstalled.env` is used only by the boost manifest: it references
the name of an environment variable that is set by the github
windows hosts and that points to a pre-built and pre-installed
copy of boost.  Since there is no package manager that we can
easily query for this sort of thing, probing from the environment
seems like a reasonable and fast way to check for this.  We
may need to evolve this over time to become more feature rich,
but this seems like a good starting point.

This commit has the potential to save 20 minutes of build time
from each public CI build just due to the boost dependency alone!

Refs: https://github.com/facebook/watchman/pull/797

Reviewed By: yfeldblum

Differential Revision: D20740410

fbshipit-source-id: 6c38019449c54465127656c3d18a6ff1f30adaea
2020-04-29 11:17:51 -07:00
..
autoconf getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
automake getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
bison getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
boost getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
cmake getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
cpptoml fbcode_builder: add manifest files for some dependencies 2019-05-03 15:59:37 -07:00
double-conversion fbcode_builder: add manifest files for some dependencies 2019-05-03 15:59:37 -07:00
eden getdeps: make several improvements to the eden manifest 2020-04-15 17:27:06 -07:00
fb303 add fbcode_builder manifest 2019-07-17 01:11:15 -07:00
fboss getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
fbthrift Remove yarpl from thrift cmake lists 2020-03-26 17:22:57 -07:00
fbthrift-rust mononoke: make mononoke_types OSS-buildable 2020-02-24 05:23:07 -08:00
fbzmq Remove dependency on fbzmq::ResourceMonitor & sigar 2020-03-04 16:37:28 -08:00
fizz Add CMake option to fetch deps statically 2019-09-12 23:35:45 -07:00
flex getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
fmt Update fmt from 5.3.0 to 6.1.1 2020-03-19 17:54:15 -07:00
folly getdeps: introduce build --enable-tests option 2019-05-08 08:21:16 -07:00
gflags fbcode_builder: add manifest files for some dependencies 2019-05-03 15:59:37 -07:00
git-lfs fbcode_builder: getdeps: fboss: add git-lfs fetcher 2019-12-03 11:43:41 -08:00
glog fbcode_builder: add manifest files for some dependencies 2019-05-03 15:59:37 -07:00
googletest fbcode_builder: add manifest files for some dependencies 2019-05-03 15:59:37 -07:00
gperf add gperf manifest 2019-09-30 12:17:08 -07:00
iproute2 fbcode_builder: update current commit for iproute2 2019-08-24 19:23:50 -07:00
katran include unit tests and integration tests in the cmake build 2019-09-18 11:58:33 -07:00
libbpf add manifest for katran and its dependencies 2019-09-18 11:58:32 -07:00
libcurl getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
libelf getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
libevent getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
libgit2 getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
libmnl getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
libnl getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
libsai Switch OSS build to SAI 1.5 2020-02-20 21:56:12 -08:00
libsodium getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
libtool getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
libusb getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
libzmq getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
lz4 getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
mononoke mononoke/configerator structs: add shipit and autocargo configs for all configerator structs 2020-04-24 04:33:53 -07:00
mvfst windows build 2019-10-31 19:03:26 -07:00
nghttp2 getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
ninja getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
OpenNSA Script to build OpenNSA kernel modules 2020-03-19 20:23:24 -07:00
openr Use BaseService for thrift-handler from oss-fb303 2020-02-27 14:36:18 -08:00
openssl getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
osxfuse fbcode_builder: add manifest files for some dependencies 2019-05-03 15:59:37 -07:00
patchelf getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
pcre getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
perl fbcode_builder: add manifest files for some dependencies 2019-05-03 15:59:37 -07:00
pexpect getdeps: add manifests for pexpect and python-ptyprocess 2019-10-01 08:44:09 -07:00
proxygen add gperf manifest 2019-09-30 12:17:08 -07:00
python getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
python-ptyprocess getdeps: add manifests for pexpect and python-ptyprocess 2019-10-01 08:44:09 -07:00
python-six add a builder that can re-package python wheel files 2019-09-19 11:33:35 -07:00
python-toml add a builder that can re-package python wheel files 2019-09-19 11:33:35 -07:00
re2 getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
rocksdb enable lz4 in the getdeps build of rocksdb 2020-02-19 15:51:57 -08:00
rsocket-cpp fbcode_builder: add manifest files for some dependencies 2019-05-03 15:59:37 -07:00
rust-shed rust toolchain: use RUSTC_BOOTSTRAP when building 2019-12-06 06:05:36 -08:00
snappy getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
sqlite3 getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
wangle getdeps: introduce build --enable-tests option 2019-05-08 08:21:16 -07:00
watchman watchman: teach watchman to talk to EdenFS on Windows 2020-03-31 12:55:23 -07:00
zlib getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00
zstd getdeps: allow satisfying deps from system packages 2020-04-29 11:17:51 -07:00