ilyakooo0/nixpkgs
1
0
Fork 0
mirror of https://github.com/ilyakooo0/nixpkgs.git synced 2024-11-13 09:17:07 +03:00
Code Issues Projects Releases Wiki Activity
552,316 Commits 96 Branches 102 Tags 4.5 GiB
babbc70adb
Commit Graph

2518 Commits

Author SHA1 Message Date
K900
b40035f472
Merge pull request #267058 from toonn/bootstrap-tools-specify-llvm 
freshBootstrapTools: Overlay the package set with the desired LLVM
 2023-11-17 14:06:37 +03:00
Adam Joseph
985ff1cb88
Update pkgs/stdenv/darwin/make-bootstrap-tools.nix 2023-11-17 08:26:41 +00:00
Weijia Wang
74d102ce50 Merge branch 'master' into staging-next 2023-11-12 19:45:20 +01:00
toonn
887d33a089 freshBootstrapTools: Overlay the package set with the desired LLVM 
As reported in #241692, since the `llvmPackages` bump the
bootstrap-tools started failing to build due to a mismatch in LLVM
versions used to build certain tools.

By overlaying the imported package set to specify `llvmPackages`, we get
everything built with the expected LLVM version.
 2023-11-12 15:08:11 +01:00
Artturin
c3c31aa798 stdenv: Improve performance 
| stat                   | before          | after           | Δ               | Δ%      |
|------------------------|-----------------|-----------------|-----------------|---------|
| cpuTime                | 513.67          | 507.77          | ↘ 5.90          | -1.15%  |
| envs-bytes             | 20,682,847,968  | 20,628,961,616  | ↘ 53,886,352    | -0.26%  |
| envs-elements          | 1,054,735,104   | 1,051,395,620   | ↘ 3,339,484     | -0.32%  |
| envs-number            | 765,310,446     | 763,612,291     | ↘ 1,698,155     | -0.22%  |
| gc-heapSize            | 53,439,602,688  | 51,711,545,344  | ↘ 1,728,057,344 | -3.23%  |
| gc-totalBytes          | 113,062,066,672 | 112,139,998,240 | ↘ 922,068,432   | -0.82%  |
| list-bytes             | 3,118,249,784   | 3,118,249,784   | 0               |         |
| list-concats           | 52,834,140      | 52,834,140      | 0               |         |
| list-elements          | 389,781,223     | 389,781,223     | 0               |         |
| nrAvoided              | 968,097,988     | 991,889,795     | ↗ 23,791,807    | 2.46%   |
| nrFunctionCalls        | 697,259,792     | 697,259,792     | 0               |         |
| nrLookups              | 510,257,062     | 338,275,331     | ↘ 171,981,731   | -33.70% |
| nrOpUpdateValuesCopied | 1,446,690,216   | 1,446,690,216   | 0               |         |
| nrOpUpdates            | 68,504,034      | 68,504,034      | 0               |         |
| nrPrimOpCalls          | 429,464,805     | 429,464,805     | 0               |         |
| nrThunks               | 1,009,240,391   | 982,109,100     | ↘ 27,131,291    | -2.69%  |
| sets-bytes             | 33,524,722,928  | 33,524,722,928  | 0               |         |
| sets-elements          | 1,938,309,212   | 1,938,309,212   | 0               |         |
| sets-number            | 156,985,971     | 156,985,971     | 0               |         |
| sizes-Attr             | 16              | 16              | 0               |         |
| sizes-Bindings         | 16              | 16              | 0               |         |
| sizes-Env              | 16              | 16              | 0               |         |
| sizes-Value            | 24              | 24              | 0               |         |
| symbols-bytes          | 2,151,298       | 2,151,298       | 0               |         |
| symbols-number         | 159,707         | 159,707         | 0               |         |
| values-bytes           | 30,218,194,248  | 29,567,043,264  | ↘ 651,150,984   | -2.15%  |
| values-number          | 1,259,091,427   | 1,231,960,136   | ↘ 27,131,291    | -2.15%  |

> Accessing the lexical scope directly should be more efficient, yes, because it changes from a binary search (many lookups) to just two memory accesses
> correction: one short linked list + one array access
> oh and you had to do the lexical scope lookup anyway for lib itself
> so it really does save a binary search at basically no extra cost

- roberth

after seeing the stats

> Oooh nice. I did not consider that more of the maybeThunk optimization becomes effective (nrAvoided). Those lookups also caused allocations!

- roberth

Left `lib.generators` and `lib.strings` alone because they're only used
once.
 2023-11-12 10:42:49 +02:00
Randy Eckenrode
940180423a
Merge pull request #265102 from reckenrode/darwin-corefoundation 
darwin.stdenv: use CoreFoundation instead of CF
 2023-11-11 12:10:10 -05:00
Alyssa Ross
506dcaabb4 stdenv: fix meson rust cross compilation 2023-11-09 10:02:24 +01:00
github-actions[bot]
fecd99b105
Merge staging-next into staging 2023-11-09 06:01:40 +00:00
github-actions[bot]
4aad65bfdb
Merge master into staging-next 2023-11-09 06:01:11 +00:00
Artturi
71f42b7f1a
Merge pull request #230874 from Ninlives/runPhase 2023-11-07 20:00:47 +02:00
Weijia Wang
8012034692 Merge branch 'staging-next' into staging 2023-11-07 03:41:49 +01:00
Artturi
fbb3ff5e12
Merge pull request #243161 from nh2/more-enableStatic 2023-11-07 03:49:21 +02:00
mlatus
707a5c09db stdenv: wrap phase running actions of genericBuild 
Provide a `runPhase` function which wraps the phase running action of
genericBuild. The new function can be used as an interface by `nix
develop`, i.e. `nix develop some#flake --build` may just call `runPhase
build`, which makes its behavior more consistent with `nix build`.
In preparation of fixing https://github.com/NixOS/nix/issues/6202
 2023-11-07 03:31:26 +02:00
Weijia Wang
023d20ae64
Merge pull request #265307 from reckenrode/clang16-fixes-batch2 
rubyPackages.iconv, v8: fix build with clang 16
 2023-11-04 17:14:36 +01:00
Randy Eckenrode
98b8d3474b
stdenvAdapters.overrideSDK: special case the 10.12 Libsystem 
The 10.12 Libsystem is not located as a sub-attribute of
`darwin.apple_sdk_10_12`. This will be fixed as part of the SDK changes
planned for post-23.11. In the meantime, special case it so the adapter
can be used to change the deployment target.
 2023-11-04 01:47:58 -04:00
Randy Eckenrode
cc4fcc147b
stdenvAdapters: add overrideLibcxx 
This was taken from #264091 to use in the interim before that PR lands
(sometime after the release of 23.11). It allows different versions of
clang to link the same libc++, allowing dependencies to be linked when
they are built with a different version of clang than the stdenv.
 2023-11-03 23:11:32 -04:00
Randy Eckenrode
daa79a1d2d
darwin.stdenv: use CoreFoundation instead of CF 
This patch switches the CoreFoundation on x86_64-darwin from the open
source swift-corelibs-foundation (CF) to the system CoreFoundation.

This change was motivated by failures building packages for the current
staging-next cycle #263535 due to an apparent incompatibility with the
rpath-based approach to choosing CF or CoreFoundation and macOS 14. This
error often manifests as a crash with an Illegal Instruction.

For example, building aws-sdk-cpp for building Nix will fail this way.

https://hydra.nixos.org/build/239459417/nixlog/1

    Application Specific Information:
    CF objects must have a non-zero isa

    Error Formulating Crash Report:
    PC register does not match crashing frame (0x0 vs 0x7FF8094DD640)

    Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
    0   CoreFoundation                	    0x7ff8094dd640 CF_IS_OBJC.cold.1 + 14
    1   CoreFoundation                	    0x7ff8094501d0 CF_IS_OBJC + 60
    2   CoreFoundation                	    0x7ff8093155e8 CFRelease + 40
    3   ???                           	       0x10c7a2c61 s_aws_secure_transport_ctx_destroy + 65
    4   ???                           	       0x10c87ba32 aws_ref_count_release + 34
    5   ???                           	       0x10c7b7adb aws_tls_connection_options_clean_up + 27
    6   ???                           	       0x10c596db4 Aws::Crt::Io::TlsConnectionOptions::~TlsConnectionOptions() + 20
    7   ???                           	       0x10c2d249c Aws::CleanupCrt() + 92
    8   ???                           	       0x10c2d1ff0 Aws::ShutdownAPI(Aws::SDKOptions const&) + 64
    9   ???                           	       0x102d9bc6f main + 335
    10  dyld                          	       0x202f333a6 start + 1942

According to a [post][1] on the Apple developer forums, hardening was
added to CoreFoundation, and this particular message occurs when you
attempt to release an object it does not recognize as a valid CF object.
(Thank you to @lilyinstarlight for finding this post).

When I switched aws-sdk-cpp to link against CoreFoundation instead of
CF, the error went away. Somehow both libraries were being used. To
prevent dependent packages from linking the wrong CoreFoundation, it
would need to be added as a propagated build input.

Note that there are other issues related to mixing CF and CoreFoundation
frameworks. #264503 fixes an issue with abseil-cpp where it propagates
CF, causing issues when using a different SDK version. Mixing versions
can also cause crashes with Python when a shared object is loaded that
is linked to the “wrong” CoreFoundation.

`NIX_COREFOUNDATION_RPATH` is supposed to make sure the right
CoreFoundation is being used, but it does not appear to be enough on
macOS 14 (presumably due to the hardening). While it is possible to
propagate CoreFoundation manually, the cleaner solution is to make it
the default. CF remains available as `darwin.swift-corelibs-foundation`.

[1]: https://developer.apple.com/forums/thread/739355
 2023-11-02 21:20:55 -04:00
Randy Eckenrode
08e1c95e9c
stdenvAdapters.overrideSDK: handle non-drv nativeBuildInputs 2023-11-01 12:30:26 -04:00
Randy Eckenrode
b6a14a3a84
stdenvAdapters.overrideSDK: also remap propagatedBuildInputs 2023-11-01 09:06:44 -04:00
Randy Eckenrode
50d046c32a
stdenvAdapters.overrideSDK: update the SDK version recursively 
Update all propagated build inputs recursively, so that propagated
inputs with propagated inputs use the correct SDK.
 2023-11-01 09:06:44 -04:00
Robert Hensing
e143a933f6 check-meta.nix: Fix flake note 
- These new-cli commands can be used with `-f`, in which case they're
  evaluated with pure evaluation disabled.
- Nix 2.4+ is not part of the condition; "flakes" is fully descriptive
  and more relatable.
- Don't suggest that it only enables this variable.
- Just don't say too much.
 2023-10-31 12:44:36 +01:00
Randy Eckenrode
3440df992e
stdenvAdapters.overrideSDK: override xcodebuild to use the requested SDK 2023-10-31 00:18:54 -04:00
toonn
f65ccb3163
Merge pull request #263598 from reckenrode/curl-propagation-fix 
curl: fix build failures due to needing to propagate frameworks
 2023-10-27 18:25:39 +02:00
github-actions[bot]
96132a216a
Merge master into staging-next 2023-10-26 18:01:02 +00:00
Randy Eckenrode
c2a623b52a
stdenvAdapters: add overrideSDK 
This is a replacement for using `darwin.apple_sdk_<ver>.callPackage`.
Instead of injecting the required packages, it provides a stdenv adapter
that modifies the derivation’s build inputs to use the requested SDK
versions. This modification extends to any build inputs propagated to it
as well. The `callPackage` approach is not deprecated yet, but it is
expected that it will be eventually.

Note that this is an MVP. It should work with most packages, but it only
handles build inputs and also only handles frameworks. Once more SDKs
are added (after #229210 is merged) and the SDK structure is normalized,
it can be extended to handle any package in the SDK namespace.

Cross-compilation may or may not work. Any cross-related issues can be
addressed after #256590 is merged.
 2023-10-26 13:26:39 -04:00
nicoo
87c22100a6 stdenv.mkDerivation: Reject MD5 hashes 
While there is no fetcher or builder (in nixpkgs) that takes an `md5` parameter,
for some inscrutable reason the nix interpreter accepts the following:
```nix
fetchurl {
  url = "https://www.perdu.com";
  hash = "md5-rrdBU2a35b2PM2ZO+n/zGw==";
}
```

Note that neither MD5 nor SHA1 are allowed by the syntax of SRI hashes.
 2023-10-25 23:14:15 +00:00
toonn
b525407492
Merge pull request #260963 from reckenrode/curl-fix 
curl: fix infinite recursion in staging-next
 2023-10-21 19:12:14 +02:00
github-actions[bot]
e5968ce788
Merge staging-next into staging 2023-10-21 06:01:24 +00:00
Artturin
1b3831f1e6 stdenv: fix typo 2023-10-21 00:26:57 +03:00
Niklas Hambüchen
e73b7f8d63 bzip2: Add enableStatic. See #61575 2023-10-20 19:59:30 +00:00
github-actions[bot]
34db746ff9
Merge staging-next into staging 2023-10-20 06:35:22 +00:00
Pavel Sobolev
17acba4cd1
stdenvAdapters.useMoldLinker: set allowedRequisites to null 2023-10-19 15:48:42 +03:00
Randy Eckenrode
df14e86d9c
stdenv.darwin: fix infinite recursion after curl update 
curl needs to link against several frameworks, but building the
frameworks (directly or indirectly) depends on curl via fetchurl and
fetchFromGitHub. Break the infinite recursion by building the SDKs’
dependencies in the last stage of the stdenv bootstrap using the prior
stage’s fetchurl and fetchFromGitHub.
 2023-10-13 18:51:05 -04:00
github-actions[bot]
c68d98e9d4
Merge staging-next into staging 2023-10-13 18:01:31 +00:00
Artturin
e64d24fff4
stdenvAdapters.useMoldLinker: automatically add allowedRequisites 
Co-Authored-By: Pavel Sobolev <paveloom@riseup.net>
 2023-10-13 10:14:40 +03:00
github-actions[bot]
49523dec15
Merge staging-next into staging 2023-10-13 06:01:45 +00:00
Artturin
77bf1395d6 mold: add tests for mold-wrapped and useMoldLinker adapter 
Make both ready for cross with prefixes

Currently
`pkgsCross.aarch64-multiplatform.mold.passthru.tests.{wrapped,adapter}`
fail with

```
Testing running the 'hello' binary which should be linked with 'mold'
Hello, world!
Checking for mold in the '.comment' section
No mention of 'mold' detected in the '.comment' section
The command was:
aarch64-unknown-linux-gnu-readelf -p .comment ...bin/hello
The output was:
String dump of section '.comment':
  [     0]  GCC: (GNU) 12.3.0
```
 2023-10-13 05:23:32 +03:00
github-actions[bot]
f1a87e41ca
Merge staging-next into staging 2023-10-09 00:03:06 +00:00
Alyssa Ross
1da4b5c99e
release.nix: namespace bootstrap tools with triples 
This will allow buliding bootstrap tools for platforms with
non-default libcs, like *-unknown-linux-musl.

This gets rid of limitedSupportSystems/systemsWithAnySupport.  There
was no need to use systemsWithAnySupport for supportDarwin, because it
was always equivalent to supportedSystems for that purpose, and the
only other way it was used was for determining which platforms to
build the bootstrap tools for, so we might as well use a more explicit
parameter for that, and then we can change how it works without
affecting the rest of the Hydra jobs.

Not affecting the rest of the Hydra jobs is important, because if we
changed all jobs to use config triples, we'd end up renaming every
Hydra job.  That might still be worth thinking about at some point,
but it's unnecessary at this point (and would be a lot of work).

I've checked by running

    nix-eval-jobs --force-recurse pkgs/top-level/release.nix

that the actual bootstrap tools derivations are unaffected by this
change, and that the only other jobs that change are ones that depend
on the hash of all of Nixpkgs.  Of the other jobset entrypoints that
end up importing pkgs/top-level/release.nix, none used the
limitedSupportedSystems parameter, so they should all be unaffected as
well.
 2023-10-07 19:38:01 +00:00
Linus Heckemann
81f6dc0864
Merge pull request #257919 from Ma27/structured-attrs-env-vars 
structured attrs: prefer `NIX_ATTRS_*_FILE` over `.attrs.*`
 2023-10-05 12:48:00 +02:00
github-actions[bot]
d4ed9dd5cd
Merge staging-next into staging 2023-10-05 00:02:58 +00:00
Maximilian Bosch
8bc5104a6e
treewide: refactor .attrs.sh detection 
When specifying the `builder` attribute in `stdenv.mkDerivation`, this
will be effectively transformed into

    builtins.derivation {
      builder = stdenv.shell;
      args = [ "-e" builder ];
    }

This also means that `default-builder.sh` is never sourced and as a
result it's not guaranteed that `$NIX_ATTRS_SH_FILE` is set to a correct
location[1].

Also, we need to source `.attrs.sh` to source `$stdenv`. So, the
following is done now:

* If `$NIX_ATTRS_SH_FILE` points to a correct location, then use it.
  Directly using `.attrs.sh` is problematic for `nix-shell(1)` usage
  (see previous commit for more context), so prefer the environment
  variable if possible.

* Otherwise, if `.attrs.sh` exists, then use it. See [1] for when this
  can happen.

* If neither applies, it can be assumed that `__structuredAttrs` is
  turned off and thus nothing needs to be done.

[1] It's possible that it doesn't exist at all - in case of Nix 2.3 or
    it can point to a wrong location on older Nix versions with a bug in
    `__structuredAttrs`.
 2023-10-04 18:36:57 +02:00
Maximilian Bosch
53c5f08415
stdenv: refactor .attrs.sh detection 
Relying on `.attrs.sh` to exist in `$NIX_BUILD_TOP` is problematic
because that's not compatible with how `nix-shell(1)` behaves. It places
`.attrs.{json,sh}` into a temporary directory and makes them accessible via
`$NIX_ATTRS_{SH,JSON}_FILE` in the environment[1]. The sole reason that
`nix-shell(1)` still works with structured-attrs enabled derivations
is that the contents of `.attrs.sh` are sourced into the
shell before sourcing `$stdenv/setup` (if `$stdenv` exists) by `nix-shell`.

However, the assumption that two files called `.attrs.sh` and
`.attrs.json` exist in `$NIX_BUILD_TOP` is wrong in an interactive shell
session and thus an inconsistency between shell debug session and actual
builds which can lead to unexpected problems.

To be precise, we currently have the following problem: an expression
like

  with import ./. {};
  runCommand "foo" { __structuredAttrs = true; foo.bar = [ 1 2 3 ]; }
    ''
      echo "''${__structuredAttrs@Q}"
      touch $out
    ''

prints `1` in its build-log. However when building interactively in a
`nix-shell`, it doesn't.

Because of that, I'm considering to propose a full deprecation of
`$NIX_BUILD_TOP/.attrs.{json,sh}`. A first step is to only mention the
environment variables, but not the actual paths anymore in Nix's
manual[2]. The second step - this patch - is to fix nixpkgs' stdenv
accordingly.

Please note that we cannot check for `-e "$NIX_ATTRS_JSON_FILE"` because
certain outdated Nix minors (that are still in the range of supported
Nix versions in `nixpkgs`) have a bug where `NIX_ATTRS_JSON_FILE` points
to the wrong file while building[3].

Also, for compatibility with Nix 2.3 which doesn't provide these
environment variables at all we still need to check for the existence of
.attrs.json/.attrs.sh here. As soon as we bump nixpkgs' minver to 2.4,
this can be dropped.

Finally, dropped the check for ATTRS_SH_FILE because that was never
relevant. In nix#4770 the ATTRS_SH_FILE variable was introduced[4] and
in a review iteration prefixed with NIX_[5]. In other words, these
variables were never part of a release and you'd only have this problem
if you'd use a Nix from a git revision of my branch from back then. In
other words, that's dead code.

[1] https://github.com/nixos/nix/pull/4770#issuecomment-834718851
[2] https://github.com/NixOS/nix/pull/9032
[3] https://github.com/NixOS/nix/issues/6736
[4] 3944a120ec
[5] 27ce722638
 2023-10-04 18:35:30 +02:00
Artturin
1270ef59e9 bootstrap-tools-cross: Rename bootstrap file jobs to be more descriptive 
Rename files in `bootstrap-files/` to match jobs in `make-bootstrap-tools-cross`.

Should make automating the bootstrap files easier.

```nix
$ nix repl --file ./pkgs/top-level/release-cross.nix
nix-repl> bootstrapTools
{ aarch64-unknown-linux-gnu = { ... }; aarch64-unknown-linux-musl = { ... }; armv5tel-unknown-linux-gnueabi = { ... }; armv6l-unknown-linux-gnueabihf = { ... }; armv6l-unknown-linux-musleabihf = { ... }; armv7l-unknown-linux-gnueabihf = { ... }; mips64el-unknown-linux-gnuabi64 = {... }; mips64el-unknown-linux-gnuabin32 = { ... }; mipsel-unknown-linux-gnu = { ... }; powerpc64le-unknown-linux-gnu = { ... }; riscv64-unknown-linux-gnu = { ... }; x86_64-unknown-linux-musl = { ... }; }
```

Additionally if non-linux bootstrap files are added to
`make-bootstrap-tools-cross` then there won't be any renaming needed.
 2023-10-03 23:34:40 +03:00
github-actions[bot]
a3a27f01d5
Merge staging-next into staging 2023-10-02 18:01:40 +00:00
Artturi
ece37b8aa4
Merge pull request #258476 from Artturin/bootstrapthingies2 2023-10-02 09:10:31 +03:00
Artturin
0848edf7a0 bootstrap-tools-cross: Add a note about what should be here 2023-10-02 01:30:11 +03:00
Artturin
46ecf91843 stdenv: remove unused loongson2f bootstrap files 
The last use of it was removed in (linux/default.nix: use mipsel.nix instead of longson.nix for mips32)[e8b10284f3].
 2023-10-02 00:35:57 +03:00
Artturin
a1c2c50cbc bootstrap-tools-cross: sort 2023-10-01 23:54:45 +03:00
Artturin
df7bf17643 bootstrap-tools-cross: remove bootstrap files for systems which are not used in stdenv/linux/default.nix 2023-10-01 23:53:15 +03:00