Summary:
Take the README.md from
7ead0e29e4/README.md
and apply it on Eden repo.
Re-add the Cargo.toml file that declares Cargo workspace.
Re-add fbcode_builder/getdeps manifest for Mononoke
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/13
Test Plan:
./build/fbcode_builder/getdeps.py build mononoke
./build/fbcode_builder/getdeps.py test mononoke
Reviewed By: ahornby
Differential Revision: D19833059
Pulled By: lukaspiatkowski
fbshipit-source-id: fb37e13306c0b9969a7c4e52b05e1a66a577022f
Summary: Mononoke is becoming part of Eden repository, remove it's getdeps manifest until we finalize the move.
Reviewed By: krallin
Differential Revision: D19769090
fbshipit-source-id: 9b471686728e9ff28317f9157f5f8a1834c5f5e4
Summary:
With this change the getdeps' CargoBuilder will support depencies between cargo builds.
The way how it works is documented in the code and required few assumptions about how a cargo project has to be defined in order to support this.
This change also adds the "mononoke" manifest and few Cargo.toml files to the mononoke project to prove that this new feature works.
Reviewed By: farnz
Differential Revision: D19468912
fbshipit-source-id: f299733cbbc2ec7bca399c898ec8d710334d0fa9
Summary:
FBOSS needs to use header files from more directories in OpenNSA,
so make those available in include directory.
Differential Revision: D19461874
fbshipit-source-id: 1fe3fbbc39477baecf9cd4f7c7a964be40cbbb12
Summary:
This is an interim solution that unblocks failing FBOSS OSS builds.
Broadcom's OpenNSA is available on github and contains large precompiled
libraries. For large files (> 100Mb), github uses git-lfs.
git clones to OpenNSA started failing yesterday with:
Error downloading object: lib/x86-64/libopennsa.a (7f21d94): Smudge error:
Error downloading lib/x86-64/libopennsa.a
(7f21d941ac32ccae38adb05b3386be739bdccd502a878364d386e7a5cb172f35): batch
response: This repository is over its data quota. Account responsible for LFS
bandwidth should purchase more data packs to restore access.
Github documents (refer:
https://help.github.com/en/github/managing-large-files/about-storage-and-bandwidth-usage)
“When you download a file tracked with Git LFS, the total file size is counted
against the repository owner’s bandwidth limit”. It appears we are hitting that
limit.
Thus, github is not a good choice here. Even if we create a 'github release
tarball', the tarball continues to carry lfs files and thus downloading those
counts towards the quota (which is limited).
Broadcom paid some amount to bump up the quota, but it is matter of time before
we hit it esp given that these builds run on-diff.
A better solution is to download a tarball (that contains real files, not lfs),
and then allow Facebook lfs to cache it.
While github does not support that, it seems bitbucket does.
Thus, we forked OpenNSA from github into a bitbucket repo and this patch
modifies the manifest to download a tarball from bitbucket that will be cached.
Broadcom is working on a cleaner solution but this will keep our FBOSS OSS build
clean in the interim.
Reviewed By: wez
Differential Revision: D19437386
fbshipit-source-id: b1213a186a0af57b9fd71b30e8899e80affd4cc0
Summary:
Broadcom has formally replaced OpenNSL with newer OpenNSA:
https://github.com/Broadcom-Network-Switching-Software/OpenNSA/blob/master/README.md
With D16401189, FBOSS no longer depends on OpenNSL, but uses the new OpenNSA
instead. Remove this manifest.
Reviewed By: wez
Differential Revision: D19413708
fbshipit-source-id: f3662f1101d59a5e0e59cb7238797d652564ff90
Summary:
With the release of OpenNSA (https://github.com/Broadcom-Network-Switching-Software/OpenNSA), we can stop using opennsal altogether!
This diff was created using following steps:
cd fboss/agent
find . ! -path "*\/oss*" -type f \( ! -iname "*TARGETS*" \) -exec sed -i -e 's/opennsl_spl_//g' {} \;
find . ! -path "*\/oss*" -type f \( ! -iname "*TARGETS*" \) -exec sed -i -e 's/opennsl/bcm/g' {} \;
find . ! -path "*\/oss*" -type f \( ! -iname "*TARGETS*" \) -exec sed -i -e 's/OPENNSL/BCM/g' {} \;
find . ! -path "*\/oss*" -type f \( ! -iname "*TARGETS*" \) -exec sed -i -e 's/OpenNSL/Bcm/g' {} \; # for symbols like snmpBcmTransmittedPkts2048to4095Octets
Furthermore:
- In all TARGETS under fboss/agent replace wrapped_opennsl_symbols with wrapped_bcm_symbols.
- hg mv agent/facebook/test/OpenNSLInterface.h agent/facebook/test/BcmInterface.h
- BcmEgress.cpp remove operator== overload for opennsl API altogether, and remove operator== BCM API version from inside unnamed namespace, and put it in fboss namespace.
- BcmCinter.h and BcmCinter.cpp now have some duplicate definitions as opennsl_ got renamed to bcm_. Delete one of the definitions "as appropriate".
- Similarly, remove duplicate definitions in FakeSdk.h and FakeSdk.cpp
- Similarly, remove duplicate definitions in SdkTracer.cpp (remove definitions calling CALL_WRAPPERS_RV_NO_CINTER). And lastly, remove CALL_WRAPPERS_RV_NO_CINTER itself.
- arc lint
- Remove OpennslCompatTests.cpp
- Fix ./agent/hw/bcm/tests/facebook/BcmEgressTest.cpp to have #include
"fboss/agent/facebook/test/MockOpenNSL.h" instead of MockBcm.h
TODO:
- This patch retains the directory structure (facebook/ subdirectory), which will be fixed in subsequent patches.
- Remove opennsl references from TARGETS file, additional cleanup etc.
- Remove all oss directories...
Differential Revision: D16401189
fbshipit-source-id: 607b6c0f97ffcacf3707f6b7a4c9454cc0b24476
Summary:
Broadcom provides this library and they decided to rename it from OpenBCM to
OpenNSA. Thus, rename corresponding fbcode_builder code.
Reviewed By: wez
Differential Revision: D19396687
fbshipit-source-id: 8233dbf4de9342b5a0e54ae275d6c73d43abe6d0
Summary: Enable `INSTALL_PYTHON_LIB=ON` for internal builds of Eden on Windows.
Reviewed By: pkaush
Differential Revision: D19060630
fbshipit-source-id: 40d4f8784958fbeaad4f0d332c55fa0cb810cb5e
Summary: The projects have been moved to a "shed/" subdirectory, so the root Cargo.toml with workspace has to be adjusted to that move.
Reviewed By: farnz
Differential Revision: D18807189
fbshipit-source-id: 0fd66fa7edd38ab4fdf905872f38fac57ae0230e
Summary:
It's impossible for a consumer of a released version of
rocksdb to do anything about this except not use it, and this particular
version of rocksdb ships with a number of shadow warnings.
Disable warning to error promotion.
See also: https://twitter.com/pcwalton/status/1201679307552083968
Reviewed By: chadaustin
Differential Revision: D18785637
fbshipit-source-id: 1db2b00b3c397d6c0b8f05b9d1c658877685c961
Summary:
OpenBCM libraries are stored with git LFS. As a result, fetcher fetches LFS pointers and not the contents. Use git-lfs to pull the real contents before copying to install dir using NoopBuilder.
In future, if more builders require git-lfs, we would consider installing
git-lfs as part of the sandcastle infra as against repeating similar
logic for each builder that requires git-lfs.
Reviewed By: wez
Differential Revision: D18759806
fbshipit-source-id: f988a0460107bc0685e7aba107daba9ed88f71e7
Summary:
As titled. This is needed by OpenBCM, and in future, possibly by other
manifests as well.
Reviewed By: wez
Differential Revision: D18759807
fbshipit-source-id: d445dfa382cea4bf96443ab9889926a4abbf0757
Summary: The cargo builder will be used to verify if an opensource Rust project passes Cargo build, test and (optionally) documentation build.
Reviewed By: markbt
Differential Revision: D18636934
fbshipit-source-id: e982e6a017eb32913e2994e7457c8add2e9d6b95
Summary:
This diff fixes `CMakeLists.txt` to enable building `openr` tests using CMake:
1. It adds `add_openr_test` CMake function that adds executable target, registers it as test, links it with bunch of libraries like GTest and GMock, etc...
2. There is no `openr/tests/OpenrModuleTestBase.cpp` anywhere in the source tree, so this commit replaces it with `openr/common/Flags.cpp`.
Reviewed By: jstrizich
Differential Revision: D18584028
fbshipit-source-id: 07d854ef98d0d2509889a08ad042a371101a2825
Summary:
Remove the Facebook-specific dependencies from the Eden manifest: these
dependencies are now available directly in the Eden repository.
Reviewed By: chadaustin
Differential Revision: D18588009
fbshipit-source-id: 590c74e38e4e05939e6955839af3ebb959f9251b
Summary:
Remove the standalone fb-mercurial-rust target that was an internal-only
dependency for the Eden build. This build step is now done entirely in the
Eden build.
Reviewed By: fanzeyi
Differential Revision: D18623943
fbshipit-source-id: c62a1155ddd1c0a6b2270c472176ba25194c6145
Summary:
- Added sai to getdeps
- added SAI to FBOSS as a dependency
- Included path of SAI headers to FBOSS's cmake file.
Reviewed By: shri-khare
Differential Revision: D18629127
fbshipit-source-id: 119a6b7f2b64bd84414d9e16a903bc8df48a35e7
Summary:
Include tools/lfs in the shipit path map for now, since it is needed to
download vendored Rust crates on internal builds. Also correctly indicate
that the fb-mercurial* dependencies also apply only to internal builds.
Reviewed By: chadaustin
Differential Revision: D18621421
fbshipit-source-id: 74f00bf318ff9aefdca8696a61a8f33587749e66
Summary: We removed this dependency in the code a while ago.
Reviewed By: saifhhasan
Differential Revision: D18366401
fbshipit-source-id: daf1c7865fe0c312694bdf395f9d7ace99f70395
Summary:
This diff allows passing a watchman version number override
via the environment as well as via the cmake `WATCHMAN_VERSION_OVERRIDE`
option.
To help invalidate the build I've added a new section to the manifest
files that allows listing out additional env vars that the project
hashes should be sensitive to. The effect of this is that we'll
re-run the cmake configure step if the listed env vars are changed.
Reviewed By: Ben0mega
Differential Revision: D17865896
fbshipit-source-id: 8ea5572b0b9b7af95ec5c310e494cb17a139ced4
Summary: This is the first step towards removing `watchman/thirdparty/tap.{cpp,h}`
Reviewed By: chadaustin
Differential Revision: D17775680
fbshipit-source-id: d6ac32c3b2489e1713fb132b0bb46d848c56811f
Summary:
We are seeing random segment fault originating from OpenSSL on macOS when
Mononoke fetching is enabled.
The cause is that on macOS we are actually linking against libcurl shipped with
the system instead of ours. That copy of libcurl is linked with macOS's
libcrypto instead of the one we compiles during Eden's build, and it seems that
version of libcrypto does not provide concurrency safety.
The solution is to build curl on macOS and make sure it is linked to our
OpenSSL that has the concurrency callbacks registered.
Reviewed By: wez
Differential Revision: D17657822
fbshipit-source-id: 85abdf3be10b3903a5efc6b3a91624c7258de790
Summary:
Add a manifest to download pexpect-4.7.0 from PyPI, as well as its ptyprocess
dependency.
Reviewed By: wez
Differential Revision: D17669618
fbshipit-source-id: 13395ec07f503f39adb3dc5aa8d0c2d8d0f1d927
Summary:
this adds `oss-openr-linux-getdeps` to diffs affecting files under openr. With soma going away and the old fbcode_builder job disabled, this will give us the signal we need to keep the cmake build healthy.
[Some Info on Getdeps](https://our.intern.facebook.com/intern/wiki/Test_your_Open_Source_build_with_getdeps.py/)
Michael, this change may require you to bump up some of the dependent libraries and build them with cmake if not already. The main changes to the cmake script are around using package configs instead of `find_library`
Also, for those with more CMake experience: since there are some big changes in the `CmakeLists`, feel free to pour on more suggestions on how I could make it better and more aligned with other facebook OSS
Reviewed By: saifhhasan
Differential Revision: D16010068
fbshipit-source-id: 66f914f1971f826e0868c4130839380639a7e44b
Summary:
Update the getdeps manifest and Eden's CMake files to indicate that the
edenfsctl program depends on python-toml.
Reviewed By: chadaustin
Differential Revision: D17401215
fbshipit-source-id: f512678d8bca9c7b2b4d25bf9c3ecd7eed825de9
Summary:
The libraries in thrift/lib/py support both Python 2 and Python 3, and rely on
the Python six module for some of this compatibility support.
Update the getdeps manifest for fbthrift to indicate this dependency, and
update fbthrift's CMakeLists.txt file to find and reference python-six
properly. This will ensure that the python-six code is built into any python
executable that uses the thrift/lib/py libraries.
Reviewed By: yfeldblum
Differential Revision: D17401218
fbshipit-source-id: 0007dda8974ae9bd87e4d7e256c74908c9a30d8f
Summary:
Add a new builder that can extract Python wheel files, and re-package them
for consumption by our add_fb_python_library() and add_fb_python_executable()
CMake functions. This is useful for dependencies on packages from PyPI.
At the moment this code only handles architecture-independent pure-Python
packages. It shouldn't be too hard to extend this to handle more complex
wheels, but for now I only need to use it for some pure-Python wheels and so I
haven't tested with more complex wheel files.
This also includes two new manifests for python-six and python-toml that take
use this new builder.
Reviewed By: wez
Differential Revision: D17401216
fbshipit-source-id: d6f74565887c3f004e1c06503dc9ec81599dd697
Summary: This enable test targets to be built and ran
Reviewed By: lnicco
Differential Revision: D17408942
fbshipit-source-id: 144d223bc3830d07a0420e9569d3166a8646cd9a
Summary: changes the way it pulls its dependencies
Reviewed By: wez
Differential Revision: D17363235
fbshipit-source-id: 37e509c7e413f763e3553e9f01ac23951045089c
Summary:
We've been squeaking by with assuming that flex is installed already
on posix systems, but that isn't the case on the github actions default
configuration.
Adjust the bison recipe: on windows it deploys both flex and bison. We use the
same source for both flex and bison but install flex to a separate install
prefix to make it easier to consume the flex dependency distinct from the bison
dependency.
The latest flex release segfaults during compilation on linux unless we
force -DGNU_SOURCE, so the manifest does that on linux.
Reviewed By: simpkins
Differential Revision: D17385051
fbshipit-source-id: 9f31b07849af9de50099d1b20bedba517bbbdf2f
Summary:
while testing https://github.com/facebook/watchman/pull/743 on macOS
I noticed that the libevent build failed to find openssl.
openssl is special on macos because apple do not ship the headers.
We already build and depend on openssl for the folly manifest on
macos, so this is really just adding a missing edge.
Reviewed By: simpkins
Differential Revision: D17385053
fbshipit-source-id: 1b688537fef422d81a959fc5749c871b9e868baa
Summary:
We would like to build a version of proxygen that has minimal
dependencies on dynamic libraries.
Reviewed By: yfeldblum
Differential Revision: D17228181
fbshipit-source-id: cfd61afdfa978c49a536184f426502196241fb8a
Summary:
This should make it easier to eg: enable fbzmq on macos at a later
time, and also makes things more explicit about what is being built, and
importantly, by listing these in the manifest ensures that we have a
hash change if we change this list; we wouldn't trigger such a change
if the list were encoded solely in builder.py
Reviewed By: chadaustin
Differential Revision: D17133149
fbshipit-source-id: caf0dd45e262188eeefafe0868ef95ad257a4950
Summary: [OpenSource] Cut `mstch` manifest, which was but is no longer used by `fbthrift`.
Reviewed By: avalonalex, davidtgoldblatt
Differential Revision: D17160816
fbshipit-source-id: 3b96142455d6990870e8ed3e5ce330d339d5a657
Summary:
boost.fiber is not available on macos and is required by fbzmq,
so disable the build on darwin.
In D17128935 I tried to make libzmq build with cmake to build
on windows, but there's a problem with its cmake and it fails
to configure.
So let's just nop out the build for !linux for now.
Reviewed By: jstrizich
Differential Revision: D17129362
fbshipit-source-id: db89010d18707a4105b8d395b1758f738336495d
Summary:
All of the requisite headers are now open sourced and eden
has been converted to use them. This removes any logic that references
this directory and kills the now unused cmake file.
Reviewed By: chadaustin
Differential Revision: D16994732
fbshipit-source-id: 573b8b50540ee64590682be2a54aa94659f12368
Summary:
Most stat code in fbcode was moved in to open source under
fbcode/fb303. The only exception that we use internally is the
MonotonicCounter class. This diff moves all of our stats code to use
the open source headers and updates cmake files to pull in the open
source fb303.
Reviewed By: chadaustin, shri-khare
Differential Revision: D16969960
fbshipit-source-id: 12b7abb54c956c242c8e27eb69fd96925c7e61f7
Summary:
Fix the fbzmq pathmap listed in its manifest file to match the path map
actually used by ShipIt. The fact that this was broken was why internal
getdeps builds did not detect that D16577367 broke the build.
Reviewed By: wez, jstrizich
Differential Revision: D17005360
fbshipit-source-id: 046ff58ad70c03b860c3fccebaba975808df244d
Summary:
The manifest file for iproute2 previously listed the commit to check out as
7ca63aef7d1b0c808da0040c6b366ef7a61f38c1. However, this is not a commit, but
a tag object that refers to commit 92a0236a3cdf3438000834121b7ea8a09f1f52b1.
This caused getdeps to think that iproute2 was always out-of-date, even when
it was already on the correct commit, since it was comparing current commit ID
against the tag ID.
This updates the manifest to refer to the commit ID instead of the tag ID.
Reviewed By: shri-khare
Differential Revision: D17005418
fbshipit-source-id: c7145c73b66ae93a2e9097ddcf8c53237b135d32
