This adds a way to specify the `evalSystem` or `evalPackages` explicitly when calling the `project` functions.
Currently if we want to make a `flake` that supports multiple systems we have few options:
* Require builders for all the supported systems (even just for `nix flake show`).
* Pass `--impure` so that haskell.nix can see `builtins.currentSystem` to set up `pkgs.evalPackages` to use that. Unfortunately this prevents nix from caching some of the work it does and often results in it recalculating for each supported system when it would otherwise be cached and take no time at all.
* Add an overlay to replace `evalPackages`. This works, but it is not straight forward.
* Materialize the nix files for the project.
This change allows `evalSystem = "x86_64-linux";` to be passed telling `haskell.nix` to run `cabal` and `nix-tools` on that system. The user will have to have a builder for that system, but does not need to have builders for the others (unless building outputs for them).
Fixes Windows cross compilation for GHC 9.0 and 9.2.
Updates wine to use the version that is included with the chosen Nixpkgs (it used to be pinned to an older version).
Moves the configuration of the GHC source into the GHC derivation. This should make adding Hadrian support easier.
Propagates library dependences (not just pkgconfig ones) on windows so that any DLLs in those libraries can be copied for TH evaluation and to the `/bin` directory of executable components.
Adds gcc and mfcgthreads as library dependencies on Windows so that the DLLs they include will be found.
Use `$pkgsHostTarget` (instead of `ghc-pkg`) to find all the DLLs can copy them to the `/bin` directory of executable components.
Adds support for __int128_t and __uint128_t to language-c to fix aarch64-darwin builds.
Fixed reinstalling packages that come with patched versions in ghcjs.
For ghc 9.0.2 Haskell.nix will use the `stm`and `filepath` source that came with `ghc` when the version desired matches. This is because 9.0.2 shipped with packages that differ from the same version in hackage.
`reinstallableLibGhc` is fixed for GHC 8.10 and above. Rather than including a patch file this works by:
* Setting the `subDir` to the `compiler` directory when building. That way relative paths to other parts of the `ghc` source tree (outside the `compiler` directory) still work.
* Files that are needed, but generated when `ghc` builds are added from the `generated` output of the ghc derivation.
`nonReinstallablePkgs` is now sorted and deduplicated before it is used (this reduces the chance of rebuilds when the list is refactored).
When downloading a `repository` block we get a number of files (not just the index). For some repositories these files will be necessary for `cabal configure` and the vanilla ones created by haskell.nix will not work. This change keeps these extra files and combines them with the main hackage index using `lndir`, so that changes to the repo will not require a new copy of the hackage index in the store.
This change also introduces `inputMap` that allows flake or niv inputs to be used to pin repository. It also works with `source-repository-package` blocks.
This is a better fix to the problem of making DLLs available to the process running in wine that is used to evaluate TH code when cross compiling for Windows.
Also fixes macOS loadArchive in TH for ghc 9
Co-authored-by: Michael Peyton Jones <michael.peyton-jones@iohk.io>
* Improve support for external Hackage repositories
This change builds #535. `repository` blocks in `cabal.project` parsed and `cabal` is used to automatically downloaded them. Then `hackage-to-nix` is used to produce the nix required.
To make it work with restricted eval (on hydra for instance) we need to include a sha256 like this:
```
repository ghcjs-overlay
url: https://input-output-hk.github.io/hackage-overlay-ghcjs
secure: True
root-keys:
key-threshold: 0
--sha256: sha256-EPlLYPmIGtxeahlOspRzwJv+60N5mqrNC2BY4jZKceE=
```
To find the correct `sha256` put in an invalid one and attempt a build.
* Add `package.buildable` option
Turning off building for a whole package requires setting `buildable = lib.mkForce false` on each of the components of the package. This change adds support for `package.buildable = false` to make it easier.
So:
```
packages.marlowe-actus.components.library.buildable = lib.mkForce false;
packages.marlowe-actus.components.exes.marlowe-actus-test-kit.buildable = lib.mkForce false;
packages.marlowe-actus.components.tests.marlowe-actus-test.buildable = lib.mkForce false;
```
Can be replaced with:
```
packages.marlowe-actus.package.buildable = false;
```
* Include dependencies of haskell.nix that were tracked in `nix/sources.json`
as flake inputs (`flake.lock` replaces `nix/sources.json`).
* Use `flake-compat` to continue to provide a compatible interface for non
flake projects.
Co-authored-by: Alexander Bantyev <balsoft@balsoft.ru>
Co-authored-by: Shea Levy <shea@shealevy.com>
This change updates to the latest `nix-tools` to get the following fixes (there are 3 PRs in nix-tools, but just the one in haskell.nix to avoid having to update the materialized files multiple times):
## Better support for source repository packages
* https://github.com/input-output-hk/nix-tools/pull/107
Currently these are replaced by the `cabalProject` functions with regular `packages:` before running cabal configure. Cabal does not treat these the same (the setting of `tests:` and `benchmarks:` in the `cabal.project` file):
* The plan found by `cabalProject` may not match the one used when running `cabal`.
* The performance of the solver may not be consistent with running `cabal`.
This change replaces `source-repository-package` with another `source-repository-package` pointing at a minimal git repo.
## Only include planned components
* https://github.com/input-output-hk/nix-tools/pull/108
Only the components in the `plan.json` are now included in the haskell.nix cabal projects. This avoids missing dependencies attempting to build components that were not in the plan. Should fix#993.
## Pick latest packages
* https://github.com/input-output-hk/nix-tools/pull/109
When the same package occurs more than once in a `plan.json` file (perhaps because it is needed both by the project itself and by one of the `setup` dependencies or `build-tool-dependencies` of the project) the latest version will now be the one picked by haskell.nix. This is a work around for a common issue with `cabal-doctest` when cross compiling to windows (an old version of Win32 is used even if a newer one was required by the projects `constraints`).
Currently if you try to build a ghcjs version that does not exist
haskell.nix falls back on the GHC version. Once ghcjs is in the
ghc tree this will work, but for now we should have an error
message that is more descriptive than attempting and failing to
build ghc for a js target.
New error looks like this:
```
$ nix-build -A pkgs-unstable.pkgsCross.ghcjs.buildPackages.haskell-nix.compiler.ghc861
error: ghcjs 8.6.1 is no longer supported by haskell.nix. Consider using 8.6.5
```
The use of `sources.HTTP` to work around issues building GHC 9.0.1 causes a mismatch between `flake` based and non
`flake` based derivations.
The fixed version of HTTP is now in hackage so updating the internal hackage index state used haskell.nix removes the need for the HTTP source override.
Anyone using `hpack` and/or `stack` will have a dependency on `internal-nix-tools` (built with a fixed version of ghc). This is fine if your `nixpkgs` is one of the ones we cache, but if not we may have to wait for an extra ghc to build. When this happens it would be nice if it is at least building a more recent version (rather than an older one).
* Add test.
* wip
* Fix merge regression
* Combine drv, drv.source and drv.dist for doctest
* Skip cabal-doctests test when cross compiling
* Add the --no-magic flag to the .cabal file for the cabal-doctests test.
This appears to be necessary on OSX.
The --no-magic flag stops the doctest executable from expanding path
arguments, trying to locate the package db, etc.
This shouldn't be necessary with cabal-doctest, since all the
necessary options to pass to doctest are computed when running the Setup.hs
script.
See
https://github.com/input-output-hk/haskell.nix/pull/427#issuecomment-605761780.
* Fix cabal-doctest support
* Skip cabal-doctest test plan cross compiling
* More fixes for cabal-doctest
* Skip cabal-doctest tests when cross compiling
Co-authored-by: Moritz Angermann <moritz.angermann@gmail.com>
Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
Co-authored-by: Hamish Mackenzie <Hamish.K.Mackenzie@gmail.com>
* Use emscripten branch of GHCJS for 8.6
* Adds release-linux-only.nix (for hydra with no mac builders)
* Sets HOME to a temp dir once per derivation to help performance of emcc
* Updates materialization
* Uses symlinks to reduce ghc derivation size for wrapped ghcjs
* Removes compiler-nix-name test
This test checked that the default compiler was overridden by the
compiler-nix-name arg. There is no default any more.
The GHCJS branches used include fixes for:
* hs$ret_1 typo (should be hs$ret1)
* A bignum issue
* Delays cabal configure errors until after evaluation when possible.
* Avoids rerunning long `cabal configure` if we already know it will fail.
* Provides a way to get a component from a package or project using a `cabal` like reference.
Code using the `tool` functions will automatically use `getComponent`.
For `(hackage-package {...}).components.library` is also ok.
```
# Consider changing hackage-package use:
(pkgs.haskell-nix.hackage-package {...}).components.exes.something
(pkgs.haskell-nix.hackage-package {...}).getComponent "exe:something"
# For any cabal project:
project.hsPkgs.somepackage.components.exes.something
project.getComponent "somepackage:exe:something"
# or do it in two steps
(project.getPackage "somepackage").getComponent "exe:something"
```
The reason for the new function is that we cannot provide the attribute interface without knowing that packages are in the project first.
Here is how the `cabal configure` error output is handled:
* The `plan-nix` derivation builds even if `cabal configure` fails.
* When it fails, it copies `failed-cabal-configure.nix` to the `$out/default.nix` along with a copy of the `cabal configure` output.
* When `failed-cabal-configure.nix` is imported and used in any way it writes the `cabal configure` output with `__trace` so it will always be visible.
* Instead of a `plan` the imported nix contains a `configurationError` pointing the `cabal configure` output.
* The intermediate functions `configurationError` and bubble it up to where it is needed.
* `getPackage` returns a mostly empty proxy for a real package when there is a `configurationError`
* The `getComponent` function always returns a derivation, but the version in the proxy writes the `cabal configure` output to stdout and calls `exit 1` (so that it will never build).
* Add `.debug` to build any component with DWARF dugug info on linux
(ghc >=8.10.2).
* Pass `enableDWARF` to `shellFor` for to get a shell where all the
components are the `.debug` ones.
`ghcOptions` has been moved from package and is now a list of strings.
old: packages.x.package.ghcOptions = "someGHCoption";
new: packages.x.ghcOptions = ["someGHCoption"];
To specify ghcOptions for all packages:
ghcOptions = ["someGHCoption"];
For a single component:
packages.x.compoents.library.ghcOptions = ["someGHCoption"];
See https://github.com/input-output-hk/nix-tools/pull/101
We should look up hsPkgs.${pkg-name}.components.exe.${component-name}
instead of including the package as a dependency.
Adding `hsPkgs.${pkg-name}` a tool it is not clear which executable in the package `haskell.nix` should choose.
Haskell.nix did not deal with this well and in fact it could lead to `nix-shell` crashing. For instance using `shellFor` to make a shell for building `haskell-language-server` with `cabal build` crashed as a dependency on `ghcide` the executable (by the `ghcide-bench` executable) caused infinite recursion. Allowing `ghcide-bench` to correctly depend just on `components.exes.ghcide` fixes this.
This PR also includes:
* Updated materialized files built with the new nix-tools and latest index-state
* Small update to GHCJS (pins happy) to fix issue that showed up when the materialization was updated
* A fix for the infinite recursion issue when updating materialized files (by adding ghc-boot-packages-unchecked).
* Performance fix for shellFor (checks just the names of component derivations when filtering).
Fixes:
Warning: `stdenv.lib` is deprecated and will be removed in the next release. Please use `pkgs.lib` instead. For more information see https://github.com/NixOS/nixpkgs/issues/108938
* Throw better error when conf files are missing
Previously, if some directories were missing .conf files, an error like
this would be thrown:
cp: missing destination file operand after '/nix/store/pnmrz06g0sa4s3yx53hgmr9k8jrh6ww0-ouroboros-network-framework-lib-ouroboros-network-framework-0.1.0.0-haddock-config/lib/ghc-8.6.5/package.conf.d'
While normally this shouldn't occur, I've seen it happen in CI anyways,
with the problem disappearing if the store path that's missing the files
was rebuilt a couple times.
With this commit, it becomes much easier to figure out which store path
is missing the files, which would be very hard to figure out from the
previous message.
* Adds support for cross package refs (with a project). Relative
directory references between packages within a project should now
work.
* Adds `includeSiblings` to `cleanSourceWith`. When `true` it
prevents the `subDir` arg from causing filtering of other directories.
* Adds `keepGitDir` to `cleanGit` to allow `.git` directory to be kept
(useful for components that use the `githash` package).
