Summary:
with the reshuffling around getdeps.py, and changes in folly/thrift/wangle,
our internal package stopped building. This diff brings it back to life. It is
a little bit interwined:
* Introduce a `--facebook-internal` switch to getdeps that causes `fb` in the
build context to be set to `on` rather than `off`. This allows specifying
options in the manifest that are FB specific.
* Introduce cmake options to control the WATCHMAN_STATE_DIR and whether cmake
should attempt to create WATCHMAN_STATE_DIR when it is installed.
* For FB specific builds on macOS, set the state dir to match our existing location
and disable creation of WATCHMAN_STATE_DIR because that will ultimately be
handling during package installation and we don't have privs for that on
the machine generating the package.
* Adjust Facebook CI scripts to use getdeps for building and testing watchman
Reviewed By: simpkins
Differential Revision: D15337970
fbshipit-source-id: 8ab3c1f810decf5f0b16337dc1cbe6dc73d26426
Summary:
The schedule type is used to differentiate between a build run during
code review (and thus runs code that isn't yet in the master branch),
from continuous or other types of run that operate on landed code.
This doesn't change any behavior yet; this diff just adds
the plumbing to pass down an optional arbitrary schedule type string
from the CI system.
In the future, we'll use the schedule type to influence the behavior
of running tests.
Reviewed By: strager
Differential Revision: D15300120
fbshipit-source-id: 3b46afef2ff171b3fa095763dd5006a54ea328b8
Summary:
The goal is to return an error code > 127 in the case of a
transient, retryable, infrastructure error. This diff generates
those in the case of failure in downloading a URL or from interacting
with LFS.
Reviewed By: strager
Differential Revision: D15266838
fbshipit-source-id: 4f52a791320123968869032c37912dded464a86e
Summary:
This controls whether tests are built or not.
They are not built by default. When `--enable-tests` is turned on,
tests are enabled for the named project only, not all of the deps.
This results in a faster build, because eg: the number of tests in
folly is very large and consumers of folly don't want to spend so
much time waiting to build tests when really all they want to do
is build their own project.
Reviewed By: strager
Differential Revision: D15246336
fbshipit-source-id: 2e22fd60c983b059f71639602712087f2a188ec6
Summary:
This is needed to correctly pick up changes made to
eg: cmake.defines sections in the manifest for first-party
projects.
Reviewed By: strager
Differential Revision: D15246337
fbshipit-source-id: 35e525e885f87d6136d5ff3b94ebf34516ab947c
Summary:
This is useful especially on Windows where the up-to-date
checks for the dependencies take a long time.
The idea is that you might run this to start:
```
$ getdeps.py build eden
```
and then while in the edit/debug/build iteration cycle:
```
$ getdeps.py build --no-deps eden
```
Reviewed By: pkaush
Differential Revision: D15200352
fbshipit-source-id: 086f2f49db967ef4d1914a69fa80067104d79136
Summary:
This prints out the installation or source prefix for a given project.
This is useful in eg: packaging scripts to figure out where they can
find the built artifacts.
Reviewed By: simpkins
Differential Revision: D14967378
fbshipit-source-id: 7e1b5de2ca7219af24cfb07b4b42de22aa410469
Summary:
previously, a relatively lame hash was computed to use
for the build directory based on some hash of the source directory.
That was good enough to get things off the ground, but in the
interest of being able to cache the build outputs and safely
invalidate them we need a slightly more rigorous implementation.
This diff computes a hash based on the manifest contents and
relevant environmental factors.
The hash is used to name the build directory which will ultimately
be cached eg: using the travis/appveyor cache directory configuration
and some other means for the FB internal CI.
The hash needs to be sufficient that we change the hash when
the manifests change. We can tolerate a false positive change
in hash (it just means that a build will take longer), but
cannot tolerate a false negative (which would result in an
incorrect build).
Reviewed By: simpkins
Differential Revision: D14710332
fbshipit-source-id: ebc2e74eafc6f3305d4412a82195bc9fb9dfa615
Summary:
This command schedules a facebook specific sandcastle job for the current
commit in your repo for each of the platforms we have support for in
sandcastle.
You can use `--dry-run` to have it print out the job specs.
To support this, I've moved around some of the support utilities
to make it easier to import them.
Reviewed By: simpkins
Differential Revision: D14710330
fbshipit-source-id: fb1e2a2ce78e52894291159514977da97028b37f
Summary:
Adds a `test` subcommand that runs the tests for project.
We're mostly interested in the 1st party facebook projects for this.
The `sandcastle` flow will run the `test` subcommand just for the "leaf" project--the one named on the command line.
Reviewed By: simpkins
Differential Revision: D14710331
fbshipit-source-id: 7d04a46cfd723894d61018de2f230140b52285ac
Summary:
This fixes a TODO; in our CI environment we want to use the
real shipit, so we'll use this flag to make that happen.
Reviewed By: simpkins
Differential Revision: D14695576
fbshipit-source-id: 64ee72c210e2472d295dcbd39c86549273b68452
Summary:
adds the command that is used to drive a build.
The dependencies of the named project are computed using
the same mechanism behind the `list-deps` subcommand.
If a manifest specifies that it uses the `cmake` builder then
a dependency on `cmake` is synthesized so that we can download
a recent version of cmake ahead of time.
The build command uses the fetcher to update the src and then
executes the builder.
Reviewed By: simpkins
Differential Revision: D14691011
fbshipit-source-id: 31ae59614651ef021a9505e89c13b5717b440071
Summary:
While the command isn't necessarily super useful
on its own, it does show that the plumbing for walking the
deps is functioning, and that is important when it comes
to building.
The output lists the projects in the order that they
would be built.
The `fetch` command has been augmented to add a `--recursive`
flag that uses the same mechanism to recursively fetch
the dependencies.
Reviewed By: simpkins
Differential Revision: D14691004
fbshipit-source-id: b00bf6ad4742f8bb0a70698f71a5fe03d6a1f453
Summary:
Adds a command that can be used to trigger a fetch for a project
```
$ ./opensource/fbcode_builder/getdeps.py fetch zstd
Cloning https://github.com/facebook/zstd.git...
---
+ git clone --depth=100 https://github.com/facebook/zstd.git /data/users/wez/scratch/dataZusersZwezZfbsource/fbcode_builder_getdeps/repos/github.com-facebook-zstd.git
Cloning into '/data/users/wez/scratch/dataZusersZwezZfbsource/fbcode_builder_getdeps/repos/github.com-facebook-zstd.git'...
remote: Enumerating objects: 3816, done.
remote: Counting objects: 100% (3816/3816), done.
remote: Compressing objects: 100% (1415/1415), done.
remote: Total 3816 (delta 2556), reused 3312 (delta 2288), pack-reused 0
Receiving objects: 100% (3816/3816), 2.93 MiB | 9.59 MiB/s, done.
Resolving deltas: 100% (2556/2556), done.
Updating /data/users/wez/scratch/dataZusersZwezZfbsource/fbcode_builder_getdeps/repos/github.com-facebook-zstd.git -> v1.3.8
---
+ git -C /data/users/wez/scratch/dataZusersZwezZfbsource/fbcode_builder_getdeps/repos/github.com-facebook-zstd.git fetch origin v1.3.8
From https://github.com/facebook/zstd
* tag v1.3.8 -> FETCH_HEAD
---
+ git -C /data/users/wez/scratch/dataZusersZwezZfbsource/fbcode_builder_getdeps/repos/github.com-facebook-zstd.git checkout FETCH_HEAD
Note: checking out 'FETCH_HEAD'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at 470344d Merge pull request #1479 from facebook/visualTest
---
+ git -C /data/users/wez/scratch/dataZusersZwezZfbsource/fbcode_builder_getdeps/repos/github.com-facebook-zstd.git submodule update --init
```
Reviewed By: simpkins
Differential Revision: D14691008
fbshipit-source-id: 3afa391360518a08ebd6ff97f5b8b4993f10c4e8
Summary:
The build options class contains some environmental and
build related information that will be passed down to fetcher
and builder objects that will be introduced in later diffs.
Reviewed By: simpkins
Differential Revision: D14691007
fbshipit-source-id: e8fe7322f590667ac28a5a3925a072056df0b3e3
Summary:
This will feed into the manifest context for system
dependent manifest sections. This is essentially the same
code borrowed from the watchman and eden getdeps.py.
Reviewed By: simpkins
Differential Revision: D14690997
fbshipit-source-id: 3d3ae146237a2cd49609aaa2bd0e785ebe21f02c
Summary:
This is the new getdeps entrypoint. It lives in `opensource/fbcode_builder` so that
it is synced into the various opensource projects at FB.
In the incarnation in this diff is has a single subcommand that can be used to validate
a manifest file.
Reviewed By: sinancepel
Differential Revision: D14690995
fbshipit-source-id: 53f8ec9bafb7b1930970079e3ce20f496ac1c188
