Commit Graph

413 Commits

Author SHA1 Message Date
Vladimír Čunát
ac3acd956f
cc-wrapper: add optional temporary hack for -B
This fixes parts in llvmPackages_{13,rocm}
e.g. build .clang for testing.
Longterm mass-rebuild fix should come in PR #225846
2023-04-12 09:37:24 +02:00
Vladimír Čunát
e2691227cd
julia{18,19,}: fix build by a temporary hack
This is a low-rebuild version of PR #225273
/cc the proper and hopefully complete fix in PR #225220
2023-04-10 16:36:55 +02:00
Vladimír Čunát
834334f2c2
Merge #169327: cc-wrapper: fix minor problems
...into staging
2023-04-04 09:25:59 +02:00
Bernardo Meurer
f1f6ca8bcd
Merge pull request #209870 from amjoseph-nixpkgs/pr/stdenv/external-gcc-bootstrap 2023-04-03 08:19:03 -07:00
Adam Joseph
7553d0fe29 stdenv: Nix-driven bootstrap of gcc
#### Summary

By default, when you type `make`, GCC will compile itself three
times.  This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.

 #### Immediate Benefits

- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
  `bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
  (i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
  This allows Nix to build stageCompare in parallel with the rest of
  nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
  derivations
- No more Frankenstein compiler: the final gcc and the libraries it
  links against (mpfr, mpc, isl, glibc) are all built by the same
  compiler (xgcc) instead of a mixture of the bootstrapFiles'
  compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.

 #### Future Benefits

- This should allow using a [foreign] `bootstrap-files` so long as
  `hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
  (lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
  offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
  built in separate (one-liner) derivations which `inherit src;`
  from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343

 #### Incorporates

- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995

 #### Closes

- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412

 #### Credits

This project was made possible by three important insights, none of
which were mine:

1. @ericson2314 was the first to advocate for this change, and
   probably the first to appreciate its advantages.  Nix-driven
   (external) bootstrap is "cross by default".

2. @trofi has figured out a lot about how to get gcc to not mix up
   the copy of `libstdc++` that it depends on with the copy that it
   builds, by moving the `bootstrapFiles`' `libstdc++` into a
   [versioned directory].  This allows a Nix-driven bootstrap of gcc
   without the final gcc would still having references to the
   `bootstrapFiles`.

3. Using the undocumented variable [`user-defined-trusted-dirs`]
   when building glibc.  When glibc `dlopen()`s `libgcc_s.so`, it
   uses a completely different and totally special set of rules for
   finding `libgcc_s.so`.  This trick is the only way we can put
   `libgcc_s.so` in its own separate outpath without creating
   circular dependencies or dependencies on the bootstrapFiles.  I
   would never have guessed to use this (or that it existed!) if it
   were not for a [comment in guix] which @Mic92 [mentioned].

My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].

[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: 5e4ec82181/gnu/packages/gcc.scm (L253)
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: 2f1948af9c/pkgs/stdenv/linux/default.nix (L380)
2023-04-02 13:49:41 -07:00
Adam Joseph
39c7885cd9 cc-wrapper: if isClang, add -L${gccForLibs.libgcc}/lib
When wrapping `clang` and using a `gccForLibs` whose `libgcc` is in
its own output (rather than the `lib` output), this commit will adds
`-L${gccForLibs.libgcc}/lib` to `cc-ldflags`.

If that flag is not added, `firefox` will fail to compile because it
invokes `clang-wrapper` with `-fuse-ld=lld` and passes `-lgcc_s` to
`lld`, but does not tell `lld` where to find `libgcc_s.so`.  In that
situation, firefox will fail to link.
2023-04-02 03:04:20 -07:00
Sergei Trofimovich
0ebe51030f cc-wrapper: make $tool-prefixed cpp wrapping unconditional 2023-03-17 18:04:54 +00:00
Sergei Trofimovich
4bdbae5ad1 cc-wrapper: wrap cpp for cross lust like to native
Without this change $target-cpp is used unwrapped and is missing
standard header search paths among other things).

Example failure:

    $ nix build -f. -L pkgsStatic.netbsd.compat
    ...
       > checking how to run the C preprocessor... x86_64-unknown-linux-musl-cpp
       > configure: error: in `/build/cvs-export/tools/compat':
       > configure: error: C preprocessor "x86_64-unknown-linux-musl-cpp" fails sanity check
       > See `config.log' for more details
2023-03-17 16:42:52 +00:00
Sergei Trofimovich
3bc81feb0d
Merge pull request #216232 from amjoseph-nixpkgs/pr/cc-wrapper/useCcForLibs
cc-wrapper: allow non-clang compilers to use gccForLibs codepath
2023-02-21 19:41:25 +00:00
Adam Joseph
6b6c06e5d7 cc-wrapper: allow non-clang compilers to use gccForLibs codepath
cc-wrapper has essentially two separate codepaths: the `gccForLibs`
codepath, used only by non-gcc (i.e. clang) compilers, and the
"other" codepath.

This PR allows non-clang compilers to opt-in to the `gccForLibs`
codepath (off by default).  To allow this, a new parameter
`ccForLibs` is exposed, since it would be extremely confusing for
gcc to be able to use `gccForLibs` but not do so by default.
2023-02-20 23:03:44 -08:00
Robert Scott
436b0d9e1f cc-wrapper: fix inverted logic around fortify & fortify3 mutual exclusion 2023-02-21 00:27:57 +00:00
Robert Scott
0eedcfc3f4
Merge pull request #212498 from risicle/ris-fortify3
hardening flags: add `FORTIFY_SOURCE=3` support
2023-02-16 21:19:30 +00:00
Felix Buehler
cdb39a86e0 treewide: use optionalString 2023-02-13 21:52:34 +01:00
wrvsrx
b59d61aa48
cc-wrapper: revert change to -isystem 2023-01-31 11:40:23 +08:00
wrvsrx
ee2b2cef35
cc-wrapper: add -cxx-isystem to libcxx-cxxflags instead of libc-cflags 2023-01-31 11:13:11 +08:00
wrvsrx
27dddbfc22
cc-wrapper: add the '-xc++' and -x*-header case to the wrapper 2023-01-31 11:12:39 +08:00
wrvsrx
d0e80df0dc
cc-wrapper: replace "-isystem" by "-cxx-isystem" for c++ header 2023-01-31 11:12:32 +08:00
Sergei Trofimovich
6b832b5332 build-support/cc-wrapper: revert "pass in non-existent --sysroot= to untangle from libc"
This reverts commit 8c80bd08b7
("build-support/cc-wrapper: pass in non-existent --sysroot= to untangle
from libc").

This change was good in spirit: we caught a few genuine problems with
`scons` based packages (`godot`, `fluxus`) and unexpected `-idirafter`
includes in various boot loadres (`ipxe`, wimboot`):

    https://github.com/NixOS/nixpkgs/pull/210004#issuecomment-1407162693

Unfortunately `--sysroot=` also has a negative impact on libary search
order for DT_NEEDED libraries and RUNPATHs of linked libraries. This
unexpectedly broke `dmd`, `d-seams`, `llvmPackages_rocm.compiler-rt`).

An interesting case of unexpected breakage is `usbmuxd2` where the bug
exposed incomplete library move on `libstdc++fs` in `gcc`.

The library breakage is very non-intuitive (on top of already unusual
layout of `cc-wrapper` driver). Let's revert this change for now.

Once it lands we can undo `--sysroot=/` workarounds merged for
`staging-next`.
2023-01-29 07:49:37 +00:00
Sergei Trofimovich
4763533cca build-support/cc-wrapper: add libstdc++fs into default library path for clang
After https://github.com/NixOS/nixpkgs/pull/210004 `usbmuxd2` started
failing to build as:

    usbmuxd2-unstable> .../ld: cannot find -lstdc++fs: No such file or directory
    usbmuxd2-unstable> clang-11: error: linker command failed with exit code 1 (use -v to see invocation)

This started happening because #210004 exposed a long-standing bug in
`gcc` derivation: `cc.lib` is missing `libstdc++fs` library:

    $ find $(nix-build --no-link -A stdenv.cc.cc.lib) | fgrep libstdc | unnix

    /<<NIX>>/gcc-11.3.0-lib/lib/libstdc++fs.la

    /<<NIX>>/gcc-11.3.0-lib/lib/libstdc++.la
    /<<NIX>>/gcc-11.3.0-lib/lib/libstdc++.so.6.0.29
    /<<NIX>>/gcc-11.3.0-lib/lib/libstdc++.so
    /<<NIX>>/gcc-11.3.0-lib/lib/libstdc++.so.6

It was not moved from `cc.out` output:

    $ find $(nix-build --no-link -A stdenv.cc.cc) | fgrep libstdc | unnix
    /<<NIX>>/gcc-11.3.0/lib/libstdc++.a
    /<<NIX>>/gcc-11.3.0/lib/libstdc++fs.a

This change adds `cc` library lookup path back to `staging-next` until
`gcc` is fixed.`
2023-01-28 07:30:33 +00:00
github-actions[bot]
c47e4b69a3
Merge master into staging-next 2023-01-25 00:02:20 +00:00
Robert Scott
00aadf0bf7 cc-wrapper: add support for fortify3 flag 2023-01-24 21:52:10 +00:00
Alyssa Ross
2ae30c9f45 llvmPackages: use libcxxrt on FreeBSD
FreeBSD doesn't use LLVM's cxxabi implementation, for backwards
compatibility reasons.  Software expects the libcxxrt API when
building on FreeBSD.  This fixes the build of
pkgsCross.x86_64-freebsd.boost.
2023-01-24 21:18:39 +00:00
Sergei Trofimovich
b9b1d958d0 Merge remote-tracking branch 'origin/staging-next' into staging
Conflicts:
	pkgs/development/libraries/qt-6/modules/qtbase.nix
	pkgs/stdenv/linux/make-bootstrap-tools.nix
2023-01-20 21:56:57 +00:00
Yureka
7a05f9de7d
gccgo: wrap go binary (#207670) 2023-01-20 14:31:54 +01:00
Sergei Trofimovich
ca4da963f8 Merge remote-tracking branch 'origin/staging-next' into staging
Conflicts:
    pkgs/development/libraries/audio/roc-toolkit/default.nix
2023-01-14 11:07:42 +00:00
Sergei Trofimovich
67a3a14f24
Merge pull request #210004 from trofi/cc-fake-sysroot
build-support/cc-wrapper: pass in non-existent --sysroot= to untangle…
2023-01-14 08:18:56 +00:00
github-actions[bot]
49722fd14a
Merge master into staging-next 2023-01-13 18:01:34 +00:00
Vladimír Čunát
befc83905c
Merge #209423: gnat12: add for x86_64-darwin 2023-01-13 18:32:21 +01:00
Martin Weinelt
c1e6c6af69 Merge remote-tracking branch 'origin/master' into staging-next 2023-01-11 03:51:33 +01:00
Adam Joseph
dfa3f14493 cc-wrapper: -march= is not allowed on powerpc
Gcc does not allow `-march=` on PowerPC:

  https://gcc.gnu.org/onlinedocs/gcc-12.2.0/gcc/RS_002f6000-and-PowerPC-Options.html#RS_002f6000-and-PowerPC-Options

Instead, `-mcpu=` should be used to set the minimum instruction set
and `-mtune=` is used to optimize instruction scheduling for a
specific processor.  Both flags take the same set of valid values,
which includes `native`.

This commit causes `isGccArchSupported` to return `false` for PowerPC
targets so we never pass an `-march=` flag, since that will always be
rejected by gcc.
2023-01-10 14:05:03 +01:00
Sergei Trofimovich
8c80bd08b7 build-support/cc-wrapper: pass in non-existent --sysroot= to untangle from libc
I would like to add an extra `gcc` build step during linux bootstrap
(https://github.com/NixOS/nixpkgs/issues/208412). This makes it early
bootstrap compiler linked and targeted against `bootstrapTools` `glibc`
including it's headers.

Without this change `gcc`'s spec files always prefer `bootstrapTools` `glibc`
for header search path (passed in as --with-native-system-header-dir=). We'can't
override it with:

- `-I` option as it gets stacked before gcc-specific headers, we need to keep
  glibc headers after gcc as gcc cleans namespace up for C standard by using
  #include_next and by undefining system macros.
- `-idirafter` option as it gets appended after existing `glibc`-includes

This `--sysroot=/nix/store/does/not/exist` hack allows us to remove existing
`glibc` headers and add new ones with `-idirafter`.

We use `cc-cflags-before` instead of `libc-cflags` to allow user to define
their own `--sysroot=` (like `firefox` does).

To keep it working prerequisite cross-symlink in gcc.libs is required:
https://github.com/NixOS/nixpkgs/pull/209153
2023-01-10 07:54:13 +00:00
Boey Maun Suang
a584587bea cc-wrapper: Wrap/link all other GNAT commands 2023-01-07 18:32:12 +11:00
Boey Maun Suang
c1308ac58c cc-wrapper: Fix gnatmake wrapper
Switches that gnatmake needs to pass to gcc must be given as
"-cargs <gcc_switches>" after at least the files to compile (see the
gnatmake docs for all the subtleties).  This commit makes that happen,
in a way that triggers rebuilds only for things that actually depend on
GNAT, and not the other compilers contained in GCC.
2023-01-07 18:32:12 +11:00
Artturin
adc8900df1 treewide: fix some core package structuredAttrs 2022-12-08 21:05:28 +02:00
Artturin
734d7df235 allow derivation attributes in env
derivations can be coerced to their output paths
2022-12-08 06:13:19 +02:00
Nick Cao
994deb2bf1 cc-wrapper: fix path to libcxx includes for cross compilers 2022-11-30 18:46:16 +01:00
John Ericson
0c09b3d17c
Merge pull request #197937 from danielbarter/cc-wrapper-hook
cc-wrapper: adding a cc-wrapper-hook to the cc-wrapper
2022-10-28 17:30:03 +01:00
Vladimír Čunát
6890557c17
Revert "Merge #191724: cc-wrapper: remove duplicate include flags"
Let's redo this later, with more testing and less bugs.
This reverts commits 4f6e99870b and d700d8e8a2.
2022-10-28 10:28:12 +02:00
Daniel Barter
77bd639c4c cc-wrapper: adding a cc-wrapper-hook to the cc-wrapper 2022-10-26 09:33:43 -07:00
Sergei Trofimovich
8480f40236
Merge pull request #191134 from Et7f3/remove-duplicate-code-cc-wrapper
cc-wrapper: remove duplicate C{C,XX}${role_post}
2022-10-16 21:02:04 +01:00
Vladimír Čunát
d700d8e8a2
stdenv cc-wrapper: revert a problematic change for non-Linux
The change from 0bea4a194f (PR #191724) broke stdenv bootstap
on *-darwin, and I didn't want to drop the *-linux binaries
that we have already.
2022-10-14 09:40:48 +02:00
Sergei Trofimovich
7d7030c6d5
Merge pull request #193871 from trofi/FORTIFY-unset-and-set
cc-wrapper/add-hardening.sh: always unset _FORTIFY_SOURCE before re-s…
2022-10-08 07:30:17 +01:00
Vladimír Čunát
4f6e99870b
Merge #191724: cc-wrapper: remove duplicate include flags
...into staging
2022-10-05 20:01:34 +02:00
Sergei Trofimovich
2a9fc04635 cc-wrapper/add-hardening.sh: always unset _FORTIFY_SOURCE before re-setting it
On darwin clang driver always sets -D_FORTIFY_SOURCE=0 under asan.
This causes -Werror to trip over macro redefinition:

    <command line>:1:9: error: '_FORTIFY_SOURCE' macro redefined [-Werror,-Wmacro-redefined]
    #define _FORTIFY_SOURCE 2
            ^

To avoid it let's always explicitly undefine it first before redefining.
2022-10-01 08:30:11 +01:00
Daniel Barter
0bea4a194f cc-wrapper/setup-hook.sh: remove duplicate flags in NIX_CFLAGS_COMPILE 2022-09-28 13:24:42 -07:00
Theodore Ni
c319d8ae3b
cc-wrapper: comment explaining C++ stdlib order
There is context here that I needed when resolving an issue in which
libc was added to NIX_CFLAGS_COMPILE before the C++ stdlib that took
me awhile to understand.

It was suggested to me that this context be included as a comment,
since it is not obvious and could help others in the future.
2022-09-20 12:14:51 +02:00
Et7f3
d937f6fc2f
cc-wrapper: remove duplicate C{C,XX}${role_post}
This might be typo but I don't know which other variable was intended to be named. This can also come from a merge conflict.
2022-09-14 02:41:05 +02:00
github-actions[bot]
efe19ddf3b
Merge staging-next into staging 2022-08-25 18:01:48 +00:00
Minijackson
bec06f5bba
cc-wrapper: disable stackprotector for MicroBlaze
This option doesn't seem to be supported in the cross-compiler
2022-08-25 16:00:49 +02:00
Theodore Ni
1f6366d38c
cc-wrapper: fortran: enable stackprotector on M1
Was failing due to https://github.com/iains/gcc-darwin-arm64/issues/54,
fix picked up in https://github.com/NixOS/nixpkgs/pull/186477.
2022-08-19 16:41:36 -07:00