* Add `package.buildable` option
Turning off building for a whole package requires setting `buildable = lib.mkForce false` on each of the components of the package. This change adds support for `package.buildable = false` to make it easier.
So:
```
packages.marlowe-actus.components.library.buildable = lib.mkForce false;
packages.marlowe-actus.components.exes.marlowe-actus-test-kit.buildable = lib.mkForce false;
packages.marlowe-actus.components.tests.marlowe-actus-test.buildable = lib.mkForce false;
```
Can be replaced with:
```
packages.marlowe-actus.package.buildable = false;
```
* Include dependencies of haskell.nix that were tracked in `nix/sources.json`
as flake inputs (`flake.lock` replaces `nix/sources.json`).
* Use `flake-compat` to continue to provide a compatible interface for non
flake projects.
Co-authored-by: Alexander Bantyev <balsoft@balsoft.ru>
Co-authored-by: Shea Levy <shea@shealevy.com>
This change updates to the latest `nix-tools` to get the following fixes (there are 3 PRs in nix-tools, but just the one in haskell.nix to avoid having to update the materialized files multiple times):
## Better support for source repository packages
* https://github.com/input-output-hk/nix-tools/pull/107
Currently these are replaced by the `cabalProject` functions with regular `packages:` before running cabal configure. Cabal does not treat these the same (the setting of `tests:` and `benchmarks:` in the `cabal.project` file):
* The plan found by `cabalProject` may not match the one used when running `cabal`.
* The performance of the solver may not be consistent with running `cabal`.
This change replaces `source-repository-package` with another `source-repository-package` pointing at a minimal git repo.
## Only include planned components
* https://github.com/input-output-hk/nix-tools/pull/108
Only the components in the `plan.json` are now included in the haskell.nix cabal projects. This avoids missing dependencies attempting to build components that were not in the plan. Should fix#993.
## Pick latest packages
* https://github.com/input-output-hk/nix-tools/pull/109
When the same package occurs more than once in a `plan.json` file (perhaps because it is needed both by the project itself and by one of the `setup` dependencies or `build-tool-dependencies` of the project) the latest version will now be the one picked by haskell.nix. This is a work around for a common issue with `cabal-doctest` when cross compiling to windows (an old version of Win32 is used even if a newer one was required by the projects `constraints`).
* factor out isSelectedComponent
* Optimise `flatLibDepends`
We no longer force the `outPath` attribute to avoid instantiating the
derivation, which is expensive
There are now 4 project modules used to check the arguments passed to the various project functions:
* `project-common.nix` - Arguments used by all the project functions
* `stack-project.nix` - Used by the `stackProject` and `stackProject'` functions
* `cabal-project.nix` - Used by the `cabalProject` and `cabalProject'` functions
* `project.nix` - Just the `projectFileName` argument that is used by `project` and `project'` functions to determine whether to call `stackProject` or `cabalProject` function.
This also includes the `rawProject.args` that was mistakenly left out of #1141 causing #1142 and improvements for the docs for the use of the `shell` argument in `flake.nix` files.
* Use nix module system to check project arguments
This change uses the nix module system for the arguments to:
* `cabalProject'`
* `cabalProject`
* `stackProject'`
* `stackProject`
This will:
* Give an error when the name or type of an argument is incorrect
(currently misnamed args are quietly ignored).
* Fixes a problem with `projectCross` not having a way to pass different args to `cabalProject`
for different platforms. We can now use:
`({pkgs, ...}: { cabalProjectLocal = if pkgs.stdenv.hostPlatform.isX then ... })`
* Add test.
* wip
* Fix merge regression
* Combine drv, drv.source and drv.dist for doctest
* Skip cabal-doctests test when cross compiling
* Add the --no-magic flag to the .cabal file for the cabal-doctests test.
This appears to be necessary on OSX.
The --no-magic flag stops the doctest executable from expanding path
arguments, trying to locate the package db, etc.
This shouldn't be necessary with cabal-doctest, since all the
necessary options to pass to doctest are computed when running the Setup.hs
script.
See
https://github.com/input-output-hk/haskell.nix/pull/427#issuecomment-605761780.
* Fix cabal-doctest support
* Skip cabal-doctest test plan cross compiling
* More fixes for cabal-doctest
* Skip cabal-doctest tests when cross compiling
Co-authored-by: Moritz Angermann <moritz.angermann@gmail.com>
Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
Co-authored-by: Hamish Mackenzie <Hamish.K.Mackenzie@gmail.com>
Currently in order to support submodules all the directories are included (but not all files) when building the whitelist. This is done because we need the `.git` files (or directories) the contain to be present when we run `git ls-files --recurse-submodules`.
Including every directory causes two performance issues:
* Creating a directory in your repo (even empty one not added to git at all) will require the whitelist to be recalculated.
* Recalculation of the whiltelist may be slowed down if there are a large number of directories. In particular `lstat` is called on every file (probably to get the right `type` to pass to the filter function).
The fix is to use `git submodule status --recursive` to get a list of all the submodule directories. Then the `.git` files (or directories) in them can be included. This has to be repeated until all the submodules are known (as `--recursive` will not see the submodules until the parent module is included).
It looks like `callCabalProjectToNix` can fail in its `cabal` call with the following error:
```
open: permission denied (Permission denied)
```
Then running the same derivation locally works. If the error is deferred there is no easy way on hydra to retry.
This change makes it so only solvers errors (which should hopefully be deterministic) are deferred. All other errors will go back to being eval time errors meaning they should be automatically retried at the next hydra eval.
* Use emscripten branch of GHCJS for 8.6
* Adds release-linux-only.nix (for hydra with no mac builders)
* Sets HOME to a temp dir once per derivation to help performance of emcc
* Updates materialization
* Uses symlinks to reduce ghc derivation size for wrapped ghcjs
* Removes compiler-nix-name test
This test checked that the default compiler was overridden by the
compiler-nix-name arg. There is no default any more.
The GHCJS branches used include fixes for:
* hs$ret_1 typo (should be hs$ret1)
* A bignum issue
* Delays cabal configure errors until after evaluation when possible.
* Avoids rerunning long `cabal configure` if we already know it will fail.
* Provides a way to get a component from a package or project using a `cabal` like reference.
Code using the `tool` functions will automatically use `getComponent`.
For `(hackage-package {...}).components.library` is also ok.
```
# Consider changing hackage-package use:
(pkgs.haskell-nix.hackage-package {...}).components.exes.something
(pkgs.haskell-nix.hackage-package {...}).getComponent "exe:something"
# For any cabal project:
project.hsPkgs.somepackage.components.exes.something
project.getComponent "somepackage:exe:something"
# or do it in two steps
(project.getPackage "somepackage").getComponent "exe:something"
```
The reason for the new function is that we cannot provide the attribute interface without knowing that packages are in the project first.
Here is how the `cabal configure` error output is handled:
* The `plan-nix` derivation builds even if `cabal configure` fails.
* When it fails, it copies `failed-cabal-configure.nix` to the `$out/default.nix` along with a copy of the `cabal configure` output.
* When `failed-cabal-configure.nix` is imported and used in any way it writes the `cabal configure` output with `__trace` so it will always be visible.
* Instead of a `plan` the imported nix contains a `configurationError` pointing the `cabal configure` output.
* The intermediate functions `configurationError` and bubble it up to where it is needed.
* `getPackage` returns a mostly empty proxy for a real package when there is a `configurationError`
* The `getComponent` function always returns a derivation, but the version in the proxy writes the `cabal configure` output to stdout and calls `exit 1` (so that it will never build).
Issues with `cabal.project.freeze` files can be tricky to debug.
While it is possible to run without a freeze file and look
at the `plan-nix` or `plan-nix.json` (the `plan.json`) created when
haskell.nix configured the project, it would be nice to
also have the output of `cabal freeze` as it will be in the
correct format and contains a different subset of `plan.json`
than the `plan-nix`.
This might also be useful for automating updating of `.freeze`
files.
This combined with setting the `use-pkg-config` flag on `postgresql-libpq` should get it compiling. There is still a problem with `openssl` that requires a `module` to fix the linking of executables.
Add the following to the `cabal.project` file or to `cabalProjectLocal`:
```
package postgresql-libpq
flags: +use-pkg-config
```
Include a module like this one to include `openssl` in the linker arguments:
```nix
{
modules = [(
{pkgs, ...}: final.lib.mkIf pkgs.stdenv.hostPlatform.isMusl {
# The order of -lssl and -lcrypto is important here
packages.postgrest.configureFlags = [
"--ghc-option=-optl=-lssl"
"--ghc-option=-optl=-lcrypto"
"--ghc-option=-optl=-L${pkgs.openssl.out}/lib"
];
})];
}
```
This PR also adds a "hackage quirk" for `postgrest` to that makes these changes for `postgrest` automatically when built it as a `tool` or with `hackage-package`.
Unfortunately we do not have a good way to avoid the need make these changes.
When a package is not in a subdirectory, but instead
in the root of the project the cleaning of the package
source was broken.
This might explain why some `cabal.project` file
based projects might also see be affected by #1013.
See https://github.com/input-output-hk/nix-tools/pull/101
We should look up hsPkgs.${pkg-name}.components.exe.${component-name}
instead of including the package as a dependency.
Adding `hsPkgs.${pkg-name}` a tool it is not clear which executable in the package `haskell.nix` should choose.
Haskell.nix did not deal with this well and in fact it could lead to `nix-shell` crashing. For instance using `shellFor` to make a shell for building `haskell-language-server` with `cabal build` crashed as a dependency on `ghcide` the executable (by the `ghcide-bench` executable) caused infinite recursion. Allowing `ghcide-bench` to correctly depend just on `components.exes.ghcide` fixes this.
This PR also includes:
* Updated materialized files built with the new nix-tools and latest index-state
* Small update to GHCJS (pins happy) to fix issue that showed up when the materialization was updated
* A fix for the infinite recursion issue when updating materialized files (by adding ghc-boot-packages-unchecked).
* Performance fix for shellFor (checks just the names of component derivations when filtering).
The builtins.fetchGit used by haskell.nix clones only the subset of a
repository corresponding to HEAD. So if we have a workflow that has a
branch "development" as HEAD, but the stack.yaml only targets commits
from "master", then it will be unable to fetch any commit.
Fixes#592
The [dependent-map](https://hackage.haskell.org/package/dependent-map)
package for instance is not clear on which free license applies to
which parts of the code, but that does not mean it is not free.
`cabal2nix` maps LicenseRef-OtherLicense to `meta.license = "unknown"`
`cabal-to-nix` and `haskell.nix` map it to a more detailed attribute
set but it includes `free = false` and that causes packages like
`dependent-map` to fail to install when `allowUnfree` has not been set.
This change removes the `free` attribute altogether, which seems more
principled than asserting that the `OtherLicense` cannot be `free`.
I think this is more correct: other package that depend on the system
odbc package may not need freetds, the problem is that the Haskell odbc
package doesn't declare its dependency on freetds.
* Adding odbc alias to the unixODBC package
* Adding odbc alias to the unixODBC package
* Adding odbc alias to the unixODBC package
* Adding comments about odbc package dependencies
* Add parser combinators to parse SPDX license expressions
The most important parser is `compoundExpression` it parses any SPDX
license expression into a list of SPDX simple-expressions.
The rationale is Nixpkgs' license metadata isn't capable of
distinguishing between the AND and OR relationships.
License exceptions aren't currently taken into account.
* Add tests for the SPDX parser combinators
I simply added a file with expressions that are expected to fail to
parse or parse successfully.
* Add the SPDX license list
The SPDX license list as attrsets following the nixpkgs lib.licenses
convention.
This uses fetchurl which is not ideal because it's not available during
pure evaluation.
It does not yet include the SPDX exceptions list.
* Refactor cabal-licenses.nix to use spdx/licenses.nix
The handling of the generic licenses is undecided still. Some have been
removed because they have better official identifiers.
* Refactor license mapping in builders
The common code in the comp- and setup-builders has been extracted and
refactored to use the SPDX expression parser.
* Use spdx-license-list-data from nixpkgs
This conveniently solves the impurity problem with using fetchurl : )
I'm not sure threading `pkgs` through everything to get access to the
spdx license list package is the right way to go about this.
* hscolour to "LGPL-2.1-only" and remove "LGPL"
* Use evalPackages for spdx and move shim to overlay
* Better fix for LGPL packages.
Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
* Adds support for cross package refs (with a project). Relative
directory references between packages within a project should now
work.
* Adds `includeSiblings` to `cleanSourceWith`. When `true` it
prevents the `subDir` arg from causing filtering of other directories.
* Adds `keepGitDir` to `cleanGit` to allow `.git` directory to be kept
(useful for components that use the `githash` package).
After investigating adding a project argument to override the
mapping of libs to packages we realised we could fix the X11
package by updating `modules/configuration-nix.nix`.
We also updated `system-nixpkgs-map.nix` to allow multiple
packages to be mapped for a single library (turned out not
to be necessary for X11).
Co-authored-by: Rodney Lorrimar <rodney.lorrimar@iohk.io>
Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
I'm wondering how/if this worked before, i couldn't find most xorg
packages on their previous path.
(Neither on the current version of nixpkgs nor on nixpkgs anno 2018)
* Coverage: allow projects with no libraries
- Prevent projects without a library from breaking the coverage report
generators.
- Added a test to ensure projects without libraries can build coverage
reports.
* Ensure the correct GHC version is used in project coverage reports
- Previously, when trying to determine the version of GHC to use for a
project coverage report, we would attempt to find the version of GHC
used in the constituent coverage reports, or default to the GHC that
is shipped with Nixpkgs. Using the GHC that ships with Nixpkgs is
problematic (specifically the coverage-no-libs tests fail to run on
Hydra), so we now ask the user to pass in the project to the
"projectCoverageReport" function so we can find the correct version of
GHC to use in a foolproof manner.
* Fix stack.yaml filtering when resolver file is in a subdirectory
`maybeCleanedSource` tries to include the resolver file into the cleaned
source, if a resolver is specified, but it was only working when the
file was in the root of the project, because all directories would be
filtered out.
* fixup! Fix stack.yaml filtering when resolver file is in a subdirectory
Fix calculating relPath for paths using subDir
* Add test case for local stack snapshot located in a subdirectory
* Remove unconditional trace for index-state
We already have messages for unusual cases (like using the latest
index-state). This message is unconditional, so quite noisy.
We include the actual index-state in the builer log so it's available if you
want to check it.
* Move source-cleaning warning to builder log
It's not *that* interesting, and someone who cares will likely check the
log.
* Move shellFor trace message to builder log
Not that interesting, really.
Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
* Drop 19.09 support
This is a year old. NixOS doesn't even support the last stable release,
let alone the one before that.
Also cuts our CI size by a third, which is always nice.
* Remove 19.09 compatibility pieces
Fixes ghcjs 8.6.5 and 8.8.4.
Uses emscripten for ghcjs 8.8 and adds scripts for building ghcjs 8.8.
Many fixes for ghcjs test failures.
Includes relocatableConfigFiles for making relocatable ghcjs bundles.
