2713083386
This patches libsigsegv to not check the stack vma on Linux, since that involves reading procfs, and we make very heavy use of sigsegv. This eliminates most of urbit's performance discrepancy between Linux and MacOS. These are the benchmarks used; note this is a local MBP vs a cloud Linux server, and the MBP is almost certainly faster hardware. We take two benchmarks, one of which decrements 10 million times and the other simply allocates 125MB of memory. These are the results: cpu-heavy == =/ n 10.000.000 |-(?~(n n $(n (dec n)))) mem-heavy == =a (bex 1.000.000.008) macos, cpu-heavy: 6 seconds macos, mem-heavy: 1 second linux-before, cpu-heavy: 30 seconds linux-before, mem-heavy: 160 seconds linux-after, cpu-heavy 9 seconds linux-after, mem-heavy 1.3 seconds This represents a 3x speedup for the cpu-heavy operation and a 120x speedup for the memory-heavy operation. This check was used to try to distinguish stack overflow from other forms of segmentation fault. In the comments in src/handler-unix.c, it describes three heuristics it uses, depending on what's available from the OS. In the linux-i386 case, all three are availble, so we simply disable the slow one. This correctly recognizes stack overflow if you simply alloca(10000000000). |
||
|---|---|---|
| .. | ||
| cmake_toolchain | ||
| linux | ||
| macos | ||
| mingw-w64 | ||
| native | ||
| pkgs | ||
| pretend_stdenv | ||
| support | ||
| .gitignore | ||
| bundle_builder.sh | ||
| default.nix | ||
| LICENSE.nixpkgs | ||
| LICENSE.txt | ||
| make_derivation.nix | ||
| pkgs.nix | ||
| README.md | ||
| top.nix | ||
nixcrpkgs
nixcrpkgs is a collection of tools for cross-compiling statically-linked, standalone software applications. With nixcrpkgs, you can specify what platforms you want to target, what libraries and build tools you depend on, and the commands that build your software. When you build your software, nixcrpkgs will automatically take care of building or retrieving everything you need, including cross-compilers and libraries.
nixcrpkgs primarily consists of Nix expressions, which are recipes for building software with Nix, the purely functional package manager. These recipes build on top of the Nix Packages collection (Nixpkgs).
Features
- Supported target platforms:
- Supported languages for cross-compiling:
- C
- C++
- Supported build platforms:
- Linux
- Supported build tools:
- Notable supported libraries:
- Qt 5.9.6
- libusb
- libusbp
- Windows API (thanks to mingw-w64)
Getting started
To get started, you should first install Nix on a Linux machine by following the instructions on the Nix website.
Next, run df -h to make sure you have enough disk space.
- The filesystem that holds
/nixshould have several gigabytes of free space. Each GCC cross-compiler takes about 300 MB while each Qt installation takes about 800 MB. - The filesystem that holds
/tmpshould have at least 4 gigabytes of free space, which will be needed while building cross-compilers. If that is not the case on your system, you can set theTMPDIRenvironment variable to tellnix-buildto perform its builds in a different directory on a filesystem with more free space.
Next, clone or download this repository and use cd to change into the
top-level directory.
To build a simple "Hello, World!" program for Windows, run:
nix-build -A win32.hello
The first time you run this command, it will take a while because Nix has to
build a cross-compiling toolchain. When nix-build is done, it will print the
name of a directory in /nix/store that holds the resulting program, and it
will create a symbolic link in the current directory named result that points
to that directory.
If you copy result/bin/hello.exe to a Windows machine and run it, you should
see a message box appear that says "Hello, World!".
If you run nix-build -A win32.hello a second time, Nix will detect that
nothing about the build recipes has changed, so it will simply print the
directory name and remake the symbolic link.
To see how the hello package is specified in nixcrpkgs, you can look in
pkgs.nix and the pkgs/hello directory. To see how the GCC cross-compiler
for Windows was specified, you can look in the mingw-w64 directory. If you
change any of the build recipes for hello or its dependencies and then run the
nix-build command again, Nix will automatically rebuild those dependencies and
anything that depends on them, ensuring that you always get a consistent build.
Obtaining the macOS SDK
If you are trying to build software for macOS, you will need to get a macOS SDK tarball and put it in the the right place. Otherwise, you will get an error like this:
error: getting attributes of path '/home/yourname/nixcrpkgs/macos/MacOSX.sdk.tar.xz': No such file or directory
To generate the tarball, follow these steps:
- On a macOS machine, install Xcode.
- Download this repository to the machine.
- In a Terminal window, run the
macos/gen_sdk_package.shscript from this repository. - After several minutes, the current directory should have a tarball with a name like
MacOSX10.12.sdk.tar.xzand a size of about 25 MB. - Copy the SDK tarball file to the machine where you will be building software,
and put it in the
macosdirectory. - The nixcrpkgs build recipe for the SDK is hardcoded to look for a file named
MacOSX.sdk.tar.xz, so rename the tarball to that. - Consider keeping a backup of the tarball so you can always rebuild any software you made with it.
Now you should be able to build your software for macOS.
Integrating nixcrpkgs into your project
The instructions above show how to cross-compile a "Hello, World!" program that
is included with nixcrpkgs. Instead of including your project in nixcrpkgs like
the hello program, you will probably want to just use nixcrpkgs as a tool in
your project. To get an idea of how to do that, you can look at other projects
that have done the same. In the projects listed below, you should look for a
file named default.nix in the top-level directory and look for build
instructions that explain what nix-build commands to run.
- The Pololu Tic Stepper Motor Controller software is a C/C++ project that uses CMake and nixcrpkgs.
- The Pololu USB AVR Programmer v2 software is a C++ project that uses CMake and nixcrpkgs.
- The Pololu USB Bootloader Utility (p-load) is a C++ project that uses CMake and nixcrpkgs.
Updating package versions
Each build recipe in nixcrpkgs specifies a version number for the software that it builds. It is relatively easy to update the recipes even if you have not worked with Nix before. The general procedure is:
- Find the build recipe you want to update. For example, if you wanted to update the version of GCC used to build Linux programs, you would update the build recipe in
linux/gcc/default.nix. - Find the part of the build recipe where the software sources are downloaded from the internet. It is usually a
fetchurlcommand with two parameters:urlandsha256. Theurlparameter usually refers to a version string defined nearby, so update that version string and/or theurlparameter as desired. - In a shell, run
nix-prefetch-url URL, where URL is the new URL specified in your modified build recipe with all version variables fully expanded). This command will download the URL you specified, store it in the Nix store, and output the hash of it in the proper format for Nix build recipes. - Update the
sha256hash string in the build recipe by replacing it with the hash that was printed in the output ofnix-prefetch-url. Updating the hash in the build recipe is important: Nix uses it to determine whether you already downloaded the right file, so if you don't update the hash then Nix might use the wrong file (e.g. an older version of the software that you downloaded earlier). - Run the usual
nix-buildcommand that you use to build your software. For example, you could go to the top-level directory of nixcrpkgs and runnix-build -A rpi.helloto build a "Hello world" program for the Raspberry Pi, or you could runnix-build -A rpi.gccto just build the cross-compiler. - Fix any error messages that happen, one at a time. (Tip: to make a
.patchfile, rundiff -ur old newwhereoldandneware directories that contain the unpatched and patched versions of the source code, respectively.) - Once things are working, consider publishing your work on Github so others can benefit from what you figured out.
Maintaining the nixcrpkgs system
You should occasionally run nix-collect-garbage to remove items that are no
longer needed and reclaim your disk space. However, note that Nix will
typically remove all of your cross compilers and libraries when you run this
command, so be prepared to do a lengthy mass rebuild. The Nix manual has more
information about Nix garbage
collection.
You should occasionally run nix-channel --update to update to the latest
version of Nixpkgs. However, when doing this, be aware that the new version of
Nixpkgs might require you to do a mass rebuild.
You should occasionally update your checkout of the nixcrpkgs repository to get the latest versions of build tools, new features, and bug fixes. Once again, this might require a mass rebuild.
If you want your builds to be very stable and reliable, you could make forks of
nixcrpkgs and/or Nixpkgs and update them at your own pace, carefully considering
any changes made by others before merging them in. That's one of the beauties
of Nix when compared to other package management systems: you will never be
forced to upgrade your build tools, and using old tools is just as easy as using
new ones. You can use the NIX_PATH environment variable to tell nix-build
to use your forked versions.
Related projects
- osxcross: Cross-compiling toolchain targeting macOS.
- musl-cross-make: Makefile-based build tool for creating cross-compilers targeting musl.
- musl_nix_arm: A fork of nixcrpkgs with a focus on building Docker images for ARM Linux.