Summary:
X-link: https://github.com/facebookincubator/velox/pull/7542
ties all of the pieces together. The bulk of the net-new logic is in `OverlayFile`, with the LMDB stuff being ported from other implementations or just delegating calls to other classes.
Reviewed By: kmancini
Differential Revision: D46914322
fbshipit-source-id: 3434b71c92ece6b94a3c08828df286b04152d50f
Summary:
Pass the include directory to the Thrift compiler to fix using the annotation library.
Fixes the following error when building openr:
```
$ opensource/fbcode_builder/getdeps.py build openr
...
[ERROR:/data/users/viz/scratch/dataZusersZvizZfbsource/fbcode_builder_getdeps/shipit/openr/openr/if/Dual.thrift:17] Could not find include file thrift/annotation/cpp.thrift
```
Reviewed By: avalonalex
Differential Revision: D45530515
fbshipit-source-id: ad5586ebe0711d4574dcb4a5e8d61ea8bb544653
Summary: Currently on latest CentOS 8 stream dev server, gflags rpm have an invalid path. fb303 will produce an invalid path in the installed cmake. Anyone using this fb303 will then fail to compile.
Reviewed By: srikrishnagopu
Differential Revision: D41073188
fbshipit-source-id: 6ad4f60391c9253be614acc5d091236bf87d4697
Summary:
If you build Watchman from source on a machine without cargo, the
error message was confusing. You'd see "command not found" somewhere
mixed into ninja's output.
Change RustStaticLibrary, which is only used by Watchman and EdenFS,
to find cargo with find_program instead of guessing its name and
assuming it exists. This gives a much less confusing error message at
CMake configure time.
Reviewed By: genevievehelsel
Differential Revision: D40441374
fbshipit-source-id: eeafe615616775c660c700e14cf1f6b2fd9715a8
Summary:
We somehow have two different RustStaticLibrary.cmake in different places (one in eden repo and the other one in the shared opensource builder).
This diff merges them and switches Eden into using the shared CMake function (for the features option).
This diff also adds the features option for rust_executable funciton, which will be used in the next diff.
Reviewed By: kmancini
Differential Revision: D39038491
fbshipit-source-id: 99d61a1d5450010b345107a9ec5c761b62004aa6
Summary: This simply adds pcre2 to be used by other projects.
Reviewed By: chadaustin
Differential Revision: D38134778
fbshipit-source-id: 15222e9c7f9e0e3a078e9d75ef55b4dbce034d8e
Summary:
Python may also be found at these paths, let's thus make sure that getdeps
built Python binaries are searching for python.dll in these paths.
Reviewed By: ikriv
Differential Revision: D37043501
fbshipit-source-id: 3de396f64b59256c1ce31c3b3da6b49e3f1f8838
Summary:
Move `FindDoubleConversion.cmake` into the shared getdeps directory, to make
it easier to share across projects. Previously folly, fizz, and wangle each
had their own copy. The versions used by fizz and wangle had a bug where they
set `double_conversion_FOUND` instead of `DoubleConversion_FOUND`.
Reviewed By: Orvid
Differential Revision: D36678744
fbshipit-source-id: 7bfba5db743cf4e1c92799faa3e0a194f0d90c95
Summary:
Call `find_package_handle_standard_args()` with a package name of `Glog`
rather than `glog`, so that it sets `Glog_FOUND` rather than `glog_FOUND`.
This argument needs to match the case of our file name (FindGlog.cmake), since
CMake variables are case-sensitive.
Reviewed By: Orvid
Differential Revision: D36678745
fbshipit-source-id: 6a0578103d12684e414664b17f7880664c0141f4
Summary:
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/110
Pull Request resolved: https://github.com/facebookexperimental/rust-shed/pull/27
Make it so that changes to rust-shed or other common rust source are used locally vendored, so they don't need to be pushed to github before they are visible in a build.
There was already some support for cargo vendoring in getdeps, but it was limited to dependencies between manifests built with cargo builder. This wasn't enough to build something like eden (cmake is main entry point, with later calls cargo) or eden_scm (make is main entry point, with later calls to cargo), so this diff adds a cargo prepare step for getdeps other primary build systems.
The cargo vendoring is done by using a cargo config file to point to the source files used by getdeps. It has two modes:
1. per crate, existing mode which is already automatic for cargo to cargo manifest dependencies. To use it for a non cargo build manifest, add crate.pathmap
2. per git url, existing mode which was only use for crates.io third-party crates, now can be enabled by setting cargo.cargo_config_file
Reviewed By: yancouto
Differential Revision: D33895469
fbshipit-source-id: 7b13c0b679532492a336ce217de875c25fe1be90
Summary:
A bunch of existing projects vendor it into their own CMake folder. But these
same projects also have fbcode_builder's CMake directory in their CMAKE_MODULE_PATH,
so move FindZstd here.
Differential Revision: D29637686
fbshipit-source-id: 805676e18f98ef217dea8511d039edc38771b529
Summary:
It looks like the various CMAKE_CXX_FLAGS_* are simply ineffective, and I'm not
sure why, setting CMAKE_CXX_FLAGS does work though, and CMake appears to add
some release/debug flags to it when generating Ninja files.
Reviewed By: fanzeyi
Differential Revision: D27862117
fbshipit-source-id: a89f6182b5bae9a087f8fcfd4c5f9526f91e2adf
Summary:
Recently I found that its impossible to access elevated Watchman daemon from non-elevated process on Windows.
```
events.js:174
throw er; // Unhandled 'error' event
^
Error: connect EPERM \\.\pipe\watchman
at PipeConnectWrap.afterConnect [as oncomplete] (net.js:1106:14)
Emitted 'error' event at:
at Socket.<anonymous> (C:\open\ovrsource\unity\socialvr\_js\node_modules\fb-watchman\index.js:118:12)
at Socket.emit (events.js:198:13)
at emitErrorNT (internal/streams/destroy.js:91:8)
at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
at process._tickCallback (internal/process/next_tick.js:63:19)
```
To fix this, it was suggested by wez to use [his library](https://github.com/wez/EleDo) to force Watchman daemon always start in normal mode on Windows. In this stack of commits I did integrated his library into project and used it to force daemon restart in normal mode when spawned from elevated terminal.
To make it happen, I checked-in library sources and created proxy project which depends on the initial library and contains header bindings and cmake configuration. I did copy pasted Rust cmake macroses from another facebook project - eden, and also created analogue of autogen.sh but for Windows - autogen.cmd.
Pull Request resolved: https://github.com/facebook/watchman/pull/878
Test Plan:
Launch elevated terminal
Start watchman.exe produced from sources
Observe daemon starting and answering
In process monitor, observe watchman.exe process running under user group
Reviewed By: fanzeyi
Differential Revision: D25595879
Pulled By: arhelmus
fbshipit-source-id: 15eb29adcf5bd4a5708b6533a1b2bacbf13f431c
Summary:
`add_fbthrift_cpp_library` should honor `BUILD_SHARED_LIBS`, and call
`add_library` with the right setting (`SHARED` if enabled, `STATIC` otherwise)
Reviewed By: yns88
Differential Revision: D24911124
fbshipit-source-id: 79df7640a758a592a3df3e9e79bb129dd57f2d47
Summary:
By putting this in `fizz-config.cmake`, we can use depend on the `sodium` target without compromising our dependents ability to find the library.
Put the search module in the common location under `fbcode_builder/CMake` to let dependents use it.
Reviewed By: yfeldblum
Differential Revision: D24686041
fbshipit-source-id: 942d1ab34feef6cadac2b584eb8cb2d999bae0ca
Summary:
Glog v0.4.0 when configured with Debug build type adds a 'd' suffix to
the library file. This results in FindGlog.cmake failing to locate it.
Update FindGlog.cmake to use check for 'glogd', and use
select_library_configurations() to set GLOG_LIBRARY to the correct
found filename.
(Note: this has no effect if a non-Debug type is used.)
Pull Request resolved: https://github.com/facebook/folly/pull/1479
Reviewed By: yfeldblum
Differential Revision: D24503510
Pulled By: Orvid
fbshipit-source-id: 705df05a4a3d7df2df8af3bb66c319fb044adbce
Summary:
Update FindGflags.cmake to work on recent CentOS and RedHat distributions.
On these distributions the CMake package configuration installed by the
gflags-devel RPM is slightly broken, and sets `gflags_INCLUDE_DIR` to a
directory that does not exist. This happens because `/lib64` is symlinked to
`/usr/lib64`, and CMake ends up searching `/lib64` before `/usr/lib64` by
default when searching for packages. Therefore it finds `gflags-config.cmake`
via the `/lib64` symlink. However, `gflags-config.cmake` computes the include
directory relative to where its config file was found, and this relative path
computation only works when using the actual `/usr/lib64` path where it was
installed. When found via `/lib64` it returns a bogus `//include` path that
does not exist.
This updates `FindGflags.cmake` to verify if the `gflags_INCLUDE_DIR` path
actually exists, and set it to a sane location instead.
This also updates the code to use the `gflags-shared` target that is exported
by the default gflags package configuration if it was only built as a shared
library.
Pull Request resolved: https://github.com/facebook/folly/pull/1409
Reviewed By: yfeldblum
Differential Revision: D23588288
fbshipit-source-id: b68a717953ae0521f568d7bcfd05ca33cd6dc578
Summary:
We use Re2 in D22877942 for parsing multiple path prefix data fetch logging,
this introduces the dependency for eden's opensource builds.
Reviewed By: chadaustin
Differential Revision: D23431175
fbshipit-source-id: 44b399e92cb89ba1403295ecd10bc8f8d769b02c
Summary:
After some experimenting, it is a little awkward if we want to specify a relative path based on the executable location. We'd need to add a bunch of path calculations to make it right, and I don't think the added complexity is really worth the effort.
As a result, let's just remove the use of relative path, and if we ever want to ship a copy of Python distribution, we can place it under the same directory as the binary.
Reviewed By: chadaustin
Differential Revision: D22394180
fbshipit-source-id: 86d27f6d16a03fe08826b5e5eafcef2a1c77997f
Summary:
In EdenFS's latest Windows package. We are seeing DLL import errors coming from `asyncio` as it requires a system native module `_overlapped.pyd`.
The underlying cause is because when we build EdenFS CLI on Sandcastle, we are linking with Python 3.6.2. The Python36.dll shipped with the EdenFS package is also coming from that version.
However, on Windows laptop. We have Python 3.6.3. Since we are not shipping the Python system libraries with us. It uses the libraries installed in the system, and it attempts to import the `_overlapped.pyd` located at `C:\Pythone36\DLLs\`. This version is compiled against Python 3.6.3, which is incompatible with the Python36.dll we are using.
----
To resolve this, we need either ship an embedded copy of Python along with EdenFS, or teach EdenFS to use the Python distribution installed in the system. This commit tweaks the executable we prepend to the archive created with zipapp to locate `Python3.dll` dynamically. This allows us to use the Python installed in the system so we can avoid the version mismatch issue.
With this setup, we can also be shipping an embedded Python version along with EdenFS, and the Python loader can look for that path. This is demonstrated with the relative DLL loading `..\python`.
In theory, we can have a package structure like this:
```
.
├── python
│ ├── ....
│ └── python3.dll
└── bin
├── ...
├── edenfsctl.exe
└── edenfs.exe
```
Reviewed By: xavierd
Differential Revision: D22325210
fbshipit-source-id: 96a3f9503e7865a5f9d95710ff13f019afcf04f1
Summary:
Previously, the Windows build was litered with warnings of the form (typo included):
warning C4250: 'C1': inherits 'C2::C2::method' via dominance
note: see declaration of 'C2::method'
Microsoft doesn't offer any recommendation, and the internet suggest that the
right `using` should silence it. That's unfortunately not the case, adding:
using C2::method
In `C1` doesn't do anything, and the compiler still complains :(.
Since the warning appears to be non-actionable, and looks more like a
"notice" than a warning, let's just silence it.
Reviewed By: wez
Differential Revision: D21395095
fbshipit-source-id: ae661b3ed61303e6361b8a15d9e7c6b9627ea8c1
Summary:
This commit resolves an issue with our zipapp executables
on Windows that meant that the only reliable way to start them was
to use the fully path to the executable.
The root cause is that the __wargv array is produced by parsing the
process command line into an array, and if you ran `watchman-wait -h`
__wargv[0] would have `watchman-wait` rather than the fully qualified
path to the executable that the zipapp plumbing requires.
The fix is to ask the system for the fully qualified path and ensure that
that gets set as both argv[0] AND argv[1].
Reviewed By: xavierd
Differential Revision: D21190350
fbshipit-source-id: eeb95084592d30a028a93b2b03877f8cc6c72729
Summary:
In coverage collection mode a special module loader is prepended to
`sys.meta_path`. In very specific conditions this module loader can end up
returning a loader pointing to a _completely wrong module_. When importing
symbols from the wrong module errors occur.
The conditions to trigger the bug are:
- running in coverage collection mode, enabling the custom loader
- the test binary is a zip (e.g. par_style=fastzip)
- having a module name where the end part matches the name of a builtin Python
module
When these conditions were met, the special loader would return the builtin
Python module instead of the expected module. E.g. when loading a module like
`myapp.somemod.platform` in a zip style binary.
The custom loader first calls `imp.find_module()` to find the module it wants
to return a wrapped loader for. This fails for modules included in the test
binary, because the builtin `imp` module can not load from zips. This was the
trigger leading to the call to the buggy code.
When the initial call to `imp.find_module()` failed, the custom loader would
try a second call, asking the internal loader to this time try any path on
`sys.path`. For most module names this call would also fail, making the custom
loader return `None`, after which Python tries other loaders on `sys.path`.
However, when the final part of the module that was asked to load matches the
name of a Python builtin module, then the second call to the `imp` module would
succeed, returning a loader for the builtin module. E.g. `platform` when asking
for `myapp.somemod.platform`.
This diff fixes the issue by removing the broken second call to the internal
loader. This will never have worked, we just have not triggered or noticed
triggering the wrong loading before.
Differential Revision: D20798119
fbshipit-source-id: dffb54e308106a81af21b63c5ee64c6ca2041920
Summary: There are downstream oss builds showing build failure caused by introduction of metadata. We want to make them proceed to build so we won't get errors in CI.
Reviewed By: iahs
Differential Revision: D19900007
fbshipit-source-id: 4201448f7980b53e407fd2bc1c74ed4ffa8e18c1
Summary: Remove the shared CheckAtomic cmake module out of the shared dir and back to the projects that need it.
Reviewed By: lukaspiatkowski
Differential Revision: D19553656
fbshipit-source-id: 5e89b5b9448ef6d6c57ef904a652e9f9a1d5dbb3
Summary:
When building with clang, the build fails with:
```
-- Looking for __atomic_is_lock_free in atomic
-- Looking for __atomic_is_lock_free in atomic - not found
CMake Error at cmake/CheckAtomic.cmake:90 (message):
Host compiler appears to require libatomic, but cannot find it.
Call Stack (most recent call first):
CMakeLists.txt:75 (include)
```
And the error is:
```
/usr/share/cmake-3.10/Modules/CheckFunctionExists.c:7:3: error: conflicting types for '__atomic_is_lock_free'
CHECK_FUNCTION_EXISTS(void);
^
<command line>:1:31: note: expanded from here
#define CHECK_FUNCTION_EXISTS __atomic_is_lock_free
^
/usr/share/cmake-3.10/Modules/CheckFunctionExists.c:7:3: note: '__atomic_is_lock_free' is a builtin with type 'int (unsigned long, const volatile void *)'
<command line>:1:31: note: expanded from here
#define CHECK_FUNCTION_EXISTS __atomic_is_lock_free
^
/usr/share/cmake-3.10/Modules/CheckFunctionExists.c:17:25: error: too few arguments to function call, expected 2, have 0
CHECK_FUNCTION_EXISTS();
~~~~~~~~~~~~~~~~~~~~~ ^
```
LLVM's CheckAtomic (https://fburl.com/bk14shjt) uses `__atomic_fetch_add_4` so I'm modifying the configs to use it as well to check for the existence of the library.
Reviewed By: yfeldblum
Differential Revision: D19497168
fbshipit-source-id: 64f77487efd16dba49055f6c4cb1cdd0fc4ae6da
Summary:
Add a `install_fb_python_executable()` function to `FBPythonBinary.cmake` for
helping to install python executables generated with
`add_fb_python_executable()`. This primarily helps by automatically looking
up the correct output file to install from the generated targets.
Reviewed By: wez
Differential Revision: D18774539
fbshipit-source-id: 4b397580d72ac448f21d1db6d2cdd653cf3635df
Summary:
On Windows, compile a small C executable to prepend to the zip file, to allow
the resulting executable files to be run directly on Windows, without needing
to explicitly invoke the Python interpreter to run the output file.
Reviewed By: wez
Differential Revision: D17733616
fbshipit-source-id: 989a93851412d0bbe1e7857aa9111db082f67a4c
Summary:
Add a function for defining Python unit tests. This creates the test
executable, and also emits logic to perform test discovery for ctest.
Reviewed By: simpkins
Differential Revision: D17610034
fbshipit-source-id: cdf15b0b04acc1d3e906a1e2a95eb327951176ba
Summary:
Export a property that indicates the path to the test executable. This is
useful for callers that want to install the binary or run it from other CMake
rules.
Reviewed By: simpkins
Differential Revision: D17647146
fbshipit-source-id: b32e2694e44a07d7c234e53a7a5c8443cb144487
Summary:
Update the code to use `os.replace()` rather than `os.rename()` so that it
won't fail on Windows if the destination path already exists.
Reviewed By: chadaustin
Differential Revision: D17462716
fbshipit-source-id: cbc06319ccb2d73868f80ab1874890ebec5a621b
Summary:
Update the thrift C++ and Python CMake rules to indicate that the output also
depends on the thrift compiler itself.
Previously the C++ rule indicated that the output depended on the thrift
template files, which caught most cases when the thrift compiler was updated,
but wasn't fully correct. The thrift templates were also removed and baked
into the thrift compiler binary in D16356056.
Reviewed By: yfeldblum, chadaustin
Differential Revision: D17401217
fbshipit-source-id: ae5cde7a7e5e07a74406a1b6f4469124187bc12f
Summary:
Add a license header to satisfy the open source linter. Use the same
header the other .cmake files have.
Reviewed By: mhlakhani
Differential Revision: D17404782
fbshipit-source-id: 66679d72c9e680f8bb8b27869e981a046b3520cf
Summary:
We would like to build a version of proxygen that has minimal
dependencies on dynamic libraries.
Reviewed By: yfeldblum
Differential Revision: D17228181
fbshipit-source-id: cfd61afdfa978c49a536184f426502196241fb8a
Summary:
On Windows we have to explicitly invoke `make_fbpy_archive.py` with python.
Therefore use CMake's built-in `FindPythonInterp` module to find the python
executable and use that when invoking `make_fbpy_archive.py`
This is slightly complicated by the effort required to find python with older
versions of CMake. We ideally still want to support versions of CMake back to
at least 3.8, which means we need to still support finding Python with the
older `FindPythonInterp.cmake` module
Reviewed By: wez
Differential Revision: D17128606
fbshipit-source-id: 3f4beff76848b8a362ebdf21198e7a8bf1b0537f
