Commit Graph

173 Commits

Author SHA1 Message Date
Hamish Mackenzie
0f7c5afc41
Add ghc 8.10.3 (#980) 2021-01-11 18:47:56 +13:00
Hamish Mackenzie
2b7d93baf6
Allow package-name:sublib-name in build-depends (#963)
With cabal 3 it should be possible to reference a sublib in the
`build-depends` of a `.cabal` file using `package-name:sublib-name`

The `cabal-sublib` test is updated to include the new type syntax
and a fix is included for the component builder.

We will need to update `plan-to-nix` in `nix-tools` as well.
For now the work around is to use a module to add the sublib to
`depends` (see `test/cabal-sublib/default.nix`).

Without this fix the `cabal-sublib:slib` was not found by
`setup configure` because only `--dependency=slib=cabal-sublib-...`
was passed.  The fix is to also pass
`--dependency=cabal-sublib:slib=cabal-sublib-...`.
2021-01-08 23:11:16 +13:00
Michael Peyton Jones
f6bc2f7315
Better 'required' job that includes everything (#976)
This requires explicitly identifying the things tht don't work and
removing them. I used our own `disabled` attribute, because `broken` is
weird and breaks evaluation, which isn't really what we want.

Cuts the number of jobs from over 2000 to 216. This seems to greatly
speeds up the time it takes small changes (where most stuff remains the
same) to be processed by hydra.

Uses https://github.com/NixOS/hydra/issues/715 to reference the
jobs in required by name avoiding OOM issues.

Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
2021-01-06 16:44:43 +13:00
Samuel Evans-Powell
ce0187b2a4
Add golden coverage test (#964)
* Add golden coverage test

- Other coverage tests check that the coverage tests can be generated,
  but don't check the coverage output itself. This commit adds a test
  to ensure that the coverage output is correct.

* Use niv and haskell.nix for hpc-coveralls

* Get `sources` from `haskell-nix.callPackages`

* Use `builtin: false` for hydra in sources.json

* Fix for cross compilation.

Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
2020-12-11 21:57:35 +13:00
Samuel Evans-Powell
4c42100df0
Coverage: allow projects with no libraries (#962)
* Coverage: allow projects with no libraries

- Prevent projects without a library from breaking the coverage report
  generators.
- Added a test to ensure projects without libraries can build coverage
  reports.

* Ensure the correct GHC version is used in project coverage reports

- Previously, when trying to determine the version of GHC to use for a
project coverage report, we would attempt to find the version of GHC
used in the constituent coverage reports, or default to the GHC that
is shipped with Nixpkgs. Using the GHC that ships with Nixpkgs is
problematic (specifically the coverage-no-libs tests fail to run on
Hydra), so we now ask the user to pass in the project to the
"projectCoverageReport" function so we can find the correct version of
GHC to use in a foolproof manner.
2020-12-10 20:47:57 +13:00
Zhenya Vinogradov
99fdad7640
Fix stack.yaml filtering when resolver file is in a subdirectory (#943)
* Fix stack.yaml filtering when resolver file is in a subdirectory

`maybeCleanedSource` tries to include the resolver file into the cleaned
source, if a resolver is specified, but it was only working when the
file was in the root of the project, because all directories would be
filtered out.

* fixup! Fix stack.yaml filtering when resolver file is in a subdirectory

Fix calculating relPath for paths using subDir

* Add test case for local stack snapshot located in a subdirectory
2020-12-09 17:43:36 +13:00
Hamish Mackenzie
b611105898
Update version number of ghc8102-experimental (#927)
Sets the version number to `8.10.2.20201118` and compiler nix name to `ghc810220201118`
2020-12-07 17:19:39 +13:00
Hamish Mackenzie
5df5be0d08
Remove pj.hsPkgs.${n} functions in favour of pj.${n} (#938)
Applies to:

 * makeConfigFiles
 * ghcWithHoogle
 * ghcWithPackages
 * shellFor
2020-11-26 19:20:10 +13:00
Hamish Mackenzie
7078b145e9
Add support for ghcjs 8.8 and making ghcjs bundles (#828)
Fixes ghcjs 8.6.5 and 8.8.4.

Uses emscripten for ghcjs 8.8 and adds scripts for building ghcjs 8.8.

Many fixes for ghcjs test failures.

Includes relocatableConfigFiles for making relocatable ghcjs bundles.
2020-11-17 22:22:53 +13:00
Joachim Breitner
bdd69693b9
Do not use evalPackages to create sourceRepo for builds (#918)
* Do not use evalPackages to create sourceRepo for builds

but still use it when creating source repo for plan calculation.

This might be a way to fix #917

* Update unit tests

* Refactor this patch, and add some comments

this is much less convoluted. I moved the fetchgit invocation from
`cabal-project-parser.nix` to `call-cabal-project-to-nix` (which makes
sense in a way, if a parser should return parsed data, and not do
stuff), and then some simplifications could happen.

Also added some comments.

* Update unit tests

* Dead code

* Less repetition in tests

* Comment tweaks

Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
2020-11-11 11:49:27 +00:00
Hamish Mackenzie
599372b4f0
Fix coverage test broken by #897 (#898) 2020-10-29 12:04:06 +13:00
Hamish Mackenzie
cda15dc5b7
Add ghc8102-plutus as a compiler-nix-name (#848)
This change introduces a `ghc8102-experimental` compiler-nix-name that behaves much like the `ghc8102` version but includes @JoshMeredith's patch `./patches/ghc/ghc-8.10-ubxt.patch` that writes the `cg_binds` field with:

```haskell
  liftIO $ registerInterfaceDataWith "ghc/phase/core" hsc_env $ \bh ->
    putWithUserData (const $ return ()) bh (map toIfaceBind $ cg_binds cg_guts)
```

Some of the places where we used to depend on the output of `plan-to-nix` containing the correct `nix-name` for the compiler have been update to avoid using incompatible interface files from the `compiler-nix-name = "ghc8102"` version of GHC.
2020-10-21 17:04:47 +13:00
Hamish Mackenzie
6134e66e1d
Fix ghc 8.10 windows cross (boot with 8.6.5) (#865)
It looks like we can't use 8.8 for booting due to
https://gitlab.haskell.org/ghc/ghc/-/issues/18143

* Adds nixpkgs 20.09 to get mingw-w64 version 6
  (also needed for ghc 8.10 windows cross compile)

* Add materialization for ghc 8.10.2 windows cross

* Fix issue with harfbuzz override so it works with nixpkgs 20.09

* Fixes eval time issue for windows index-state test (turns off native-dns cabal flag for windows)

* Include exe extension for windows in coverage test

lib:ghc is still broken for ghc 8.10 (all variants), but it always has been and we should try to fix it properly, but it will be very tricky.
2020-10-07 15:42:32 +13:00
Samuel Evans-Powell
48b8674f5f
Add support for coverage (#762)
- Added the ability to generate coverage reports for packages and
  projects.
  - Outputs mix and tix information, as well as a HTML report.
- Added the "doCoverage" module option that allows users to choose
  packages to enable coverage for.
- Added a "doCoverage" flag to the component builder that outputs HPC
  information when coverage is enabled.
- Added the "overrideModules" library function to make it more
  ergonomic fo users to enable coverage on existing projects.
- Modified the "check" builder to also output ".tix" files (if they
  exist). This information is required to generate the coverage
  report.
- Added a test for coverage.
2020-09-29 11:56:24 +08:00
Hamish Mackenzie
e9de8c973e
Fix issue where ifdLevel with HLS tests (#849)
When building a new GHC in haskell.nix it is often desirable to set the
ifdLevel to make sure tests that use eval time do not run.
Unfortunately when ifdLevel=2 eval time work for the Haskell Language
Server tests runs and builds nix-tools for the first time at eval time.

This is fixed by separating the building of the HLS from the
ifdInputs that are needed for the building.
2020-09-17 23:20:55 +12:00
John A. Lotoski
09526c8555
Spelling, typo and whitespace fixes (#833)
* Spelling and typo fixes in doc and code comments
* Trailing whitespace or whitespace only truncation
* readTheDocs formatting corrections
2020-08-31 11:08:25 +12:00
Hamish Mackenzie
b368a5dfb4
Cache ghc883 and ghc8101 again (without tests) (#821)
When ghc 8.8.4 and ghc 8.10.2 were added we stopped building the older
versions on ci.  This changes adds them back but does not run all
the tests on them.
2020-08-15 00:34:32 +12:00
Hamish Mackenzie
c7c7d6c43a
Add ghc 8.10.2 (#807) 2020-08-11 14:26:50 +12:00
Hamish Mackenzie
85609feac4
Improve cabal.project parsing (#802)
Currently adding comments to a `source-repository-package` breaks the
parser.
2020-08-06 14:59:26 +12:00
Hamish Mackenzie
251c899804
Fix use of multiple subdirs in cabal.project (#790)
It is now possible to specify multiple subdirs in a
`source-repository-package` block.  This change fixes the parser
code in haskell.nix so that it understands.
2020-07-28 14:19:58 +12:00
Hamish Mackenzie
b6de6ef3b8
Remove components.all (#776)
It causes a lot of issues. To make this work:

* `shellFor` is updated to use `getAllComponents` instead of `.all`.
* `getAllComponents` is updated to work on the package rather than
  the package config.
* Tests updated to not use `.library` or `.exes.X` where appropriate.
* Documentation updated.
* Out of date examples removed.

As a bonus `shellFor` now takes a `components` argument that might be
useful for limiting the dependencies of the shell to just the ones
needed for the components you intend to work on.
2020-07-21 16:06:54 +12:00
Hamish Mackenzie
cacfba0b45
Replace ghc 8.8.3 with 8.8.4 (#766)
Also adds some tooling/docs for adding new ghc versions (see `docs/adding-new-ghc.md`)
2020-07-21 14:10:23 +12:00
Hamish Mackenzie
4cac8bd00f
Remove internal deps on default ghc and stackage (#738)
Changes to the interface of haskell.nix (from the changelog.md file):

* Removed `sources.nixpkgs-default`, use `sources.nixpkgs` instead.
* Removed `./nixpkgs` directory, use  `(import ./. {}).sources`
  or `./nix/sources.nix` instead.
* Removes V1 interface for details on how to fix old code see:
    https://github.com/input-output-hk/haskell.nix/issues/709
* Removed defaultCompilerNixName.
* cabalProject, cabalProject', hackage-project and hackage-package
  now require a `compiler-nix-name` argument.
* `haskell-nix.tool` and `.tools` now require a `compiler-nix-name` argument.
  New functions `p.tool` and `p.tools` (where p is a project) do not.
  Like `shellFor { tools = ... }` they will use the compiler nix name
  from the project (including stack projects where it is derived from
  the resolver).
* `haskell-nix.alex` and `haskell-nix.happy` have been removed. Use
  `p.tool "alex" "3.2.5"` or `shellFor { tools = { alex = "3.2.5"; } }`.
* `haskell-nix.nix-tools` -> `haskell-nix.nix-tools.ghc883` (it includes
  the hpack exe now).
* `haskell-nix.cabal-install` -> 
  `p.tool "cabal" "3.2.0.0"` or `shellFor { tools = { cabal = "3.2.0.0"; } }`
* `haskell-nix.haskellNixRoots` -> `haskell-nix.roots ghc883` or `p.roots`

Other changes:

Adds hpack executable to the nix-tools derivations.

Adds a `cabal-hpack` test to make sure `hpack` works with
`cabalProject`.

Reduces the number of calls to `cabalProject` (particularly when
checking materialization), by giving internal tools a per-compiler
attribute.

Uses happy 1.19.12 when building newer ghc versions.

Updates cabal-install 3.2.0.0 to use the source from github that
is compatible with ghc 8.10.1.

Updates the docs for callCabalProjectToNix.

Adds a license mapping to fix a common warning.
2020-07-08 22:54:01 +12:00
Hamish Mackenzie
c7736060cc
download stack resolver if a sha256 is provided (#748)
This change allows a resolverSha256 to be passed to stackProject.
When provided the resolver found in the stack.yaml is fetched
using this sha256 and the uri string is replaced with the
path to the fetched version.
2020-07-04 14:35:52 +12:00
Hamish Mackenzie
1e3c1304c1
Split haddock out into its own derivation (#725)
Generation the Haddock documentation for a component in a dedicated
derivation (instead of as part of the derivation to build the component).
This means that the haddock documents will not be built until they are
required for instance in a shellFor shell.

Co-authored-by: Richard Wallace <richard.wallace@simspace.com>
2020-06-27 01:29:41 +12:00
Moritz Angermann
a500f39da7
Musl changes (#700)
This should simplify the interaction with musl quite a but.
Musl specific mapping should go into the musl overlay now.
2020-06-23 09:45:22 +08:00
Hamish Mackenzie
9e6820cc71
Add haskell-nix.project (and project') (#703)
Generalized version of the `cabalProject` and `stackProject`
functions.

Automatically selects the correct project type based on the
`projectFileName` argument if provided or what files exist in
the `src` directory if no `projectFileName` was specified.

If it cannot be determined which it should use the following
error message is given:

```
error: haskell-nix.project : both `stack.yaml` and `cabal.project` files exist set `projectFileName = "stack.yaml;"` or `projectFileName = "cabal.project";`
```
2020-06-19 20:07:19 +12:00
Hamish Mackenzie
d98b1e3b6e
Add sha256map to cabalProject & stackProject (#697)
This is a simple way to specify the sha256 values for references
to git repositories that do not have a `--sha256:` comment in the
`cabal.project` file or a `# nix-sha256:` comment in the `stack.yaml`
file.

The tests demonstrate how to use `sha256map` with both
`cabalProject` and `stackProject` to build the
haskell-language-server from github with the sha256 hashes
needed to work with `--option restrict-eval true`.
2020-06-16 13:40:19 +12:00
Hamish Mackenzie
dcadfa8ab4
Use suitable hackage state for index-state (#673)
Fixes #672 by using the first index-state that is > than desired
index state.  It also passes `--index-state` to `cabal v2-cconfigre`
to make sure it cabal uses only the packages in the hackage
file that are before the specified date.

Fixes #540 by returning the `index-state` used back out of
`cabalProject` in the same way `plan-nix` is returned.
2020-06-13 01:03:17 +12:00
Hamish Mackenzie
15fc6b3774
Add ghc 8.10.1 (#541)
* ghc 8.10
* Adds/Updates/Removes materialization files
* Disable one-shot-kqueue-on-macos patch on ghc-8.10
* Test ghc883 and ghc8101 but only with nix 20.03
* Various smaller fixes.

This should fix the infinite recursion issue with #654

Co-authored-by: Lennart Spitzner <hexagoxel@hexagoxel.de>
2020-06-07 11:05:44 +08:00
Hamish Mackenzie
ff9a01e82e
Add missing pins and test ghc865 and ghc883 (#641)
Now that the defaultCompilerNixName is used to choose which GHC to use
for tools we need to update haskellNixRoots to make sure the common
tools wind up in the cache.

To do this we add the compilerNixName as a dimension in ci.nix.
This has the added bonus of ensuring the tests a run with the
supported versions of ghc (not just the default one).

haskellNixRoots is also updated to include the tools built with the
defaultCompilerNixName (in particular cabal-install was not pinned
and this was the likely cause of significant builds due to cache
misses).
2020-06-02 11:44:03 +12:00
Hamish Mackenzie
d43f6fcdd6
Fix compiler-nix-name test for mingwW64 (#613) 2020-05-17 17:08:03 +12:00
Hamish Mackenzie
788e198376
Add compiler-nix-name to cabalProject (#610)
* Add compiler-nix-name to cabalProject

Setting the compiler explicitly does not make it easy for cross
compilation to also choose the correct compiler when compiling
build tools for the build platform.  Also it is easy to pass the
wrong compiler type.  Specifying just the nix-name of the compiler
makes that less likely.

Co-authored-by: Moritz Angermann <moritz.angermann@gmail.com>
2020-05-16 21:48:48 +08:00
Moritz Angermann
b2b88f567f
Update default.nix 2020-05-02 19:24:52 +08:00
Moritz Angermann
642b2b12bf
Update tests.sh 2020-05-02 19:06:54 +08:00
Maxim Koltsov
07031ee224
External hackages (#535)
* Add support for external Hackage repositories

Currently haskell.nix is not able to build Cabal projects that depend on
packages from private Hackage repositories, as it make only main Hackage
available to cabal. This is unfortunate.

This commit adds this functionality, by allowing the user to pass
`extra-hackages` and `extra-hackage-tarballs` to `mkPkgSet` and
`callCabalToNix` respectively, to add as much extra repositories as
needed.

This repositories are first made available to Cabal when calling
`v2-configure`, resulting in correct plans. Later they are combined with
global Hackage when building dependencies of the local packages.

* Use cabal.project.freeze if available

Currently callCabalProjectToNix does not copy `cabal.project.freeze`
from source directory, leading to different build plans when building
components with nix and when building project with `cabal new-build`
inside `nix-shell`.

This behavior is undesired, so this commits fixes it.

* Add tests for extra-hackages functionality

Co-authored-by: Moritz Angermann <moritz.angermann@gmail.com>
2020-05-02 18:16:34 +08:00
Hamish Mackenzie
52401d7cb8
Fix quirks with overlays/hackage-default-args.nix (#564)
Adds a place for fixes to common problems that arise when using
`hackage-project` and `hackage-package`.
2020-04-29 13:20:20 +12:00
Hamish Mackenzie
829cba55bc
Fix lookup-sha256 test for musl and windows cross (#558) 2020-04-24 13:57:36 +12:00
Hamish Mackenzie
1f94c06e67
Use fetchGit for source-repo-package without sha (#554)
* Downloads using `builtins.fetchGit` when no sha is found.

* Adds a `lookupSha256` arg for when we can't change `cabal.project`.

* Outputs a message when the sha256 is missing that includes advice on how add one.
2020-04-21 00:10:54 +12:00
Hamish Mackenzie
d1102d342c
Add pins and materialization for ghc-boot-packages and ghc-extra-packages (#546)
* Include `tests` in path for cross compiled tests

* Add ghc-boot-packages-nix to haskellNixRoots

* Fix naming warnings in tests sources

* Uses a single cleanGit call for all the tests

* Add memoize code for ghc-boot and ghc-extra pkgs

* Memoize ghc-boot and ghc-extra pkgs nix

* Fix release.nix being more strict than ci.nix by updating filterAttrsOnlyRecursive in ci-lib.nix

* Nicer errors when materialized path does not exist

* Updated materialization docs

* Add internalHackageIndexState to set the index-state used within haskell.nix
2020-04-20 13:27:52 +12:00
Michael Peyton Jones
eb15db0450
Expose our nixpkgs pins by restructuring default.nix (#514)
* Remove unused (and broken) test files

* Make default.nix expose pinned nixpkgs

Also make `nixpkgs/default.nix` just an attribute set of sources.
This is simpler, and consistent with what e.g. `niv` does.

* Add backwards compatibility shim, and version argument to allow evolution in future

* Fix some missed things

* Fix and improve quickstart
2020-03-31 09:12:40 +08:00
Michael Peyton Jones
64700ae79e
CI: share between ci.nix and release.nix (#504)
* Define release.nix in terms of ci.nix

Beef up ci.nix to handle the same things as release.nix, then implement
the one in terms of the other.

* Fix infinite loop in evaluating release.nix with high ifdLevel

* Add tests to cross-compile

* Remove obsolete fix

* Do generic platform filtering

* Undefined variable

* Extract ci-lib.nix

* More undefined variables

* Use filterAttrsOnlyRecursive to avoid forcing all the drv attributes

* Set ifdLevel 0 to see if it fixes hydra eval

* Turn off tests on aarch64 cross

* Ifd level 1

* Ifd level 2

* Ifd level 3

* Maybe it's required that's too big?

Co-authored-by: Hamish Mackenzie <Hamish.Mackenzie@iohk.io>
2020-03-25 11:20:58 +00:00
Michael Peyton Jones
171c57ab2c
Add 'name' to cleanSource calls in tests (#505)
Otherwise we get unnecessary rebuilding of the tests in CI.
2020-03-22 23:52:10 +00:00
Rodney Lorrimar
0933c58908
Fix evaluation of haskellLib.collectComponents (#479) 2020-03-09 20:47:26 +10:00
Hamish Mackenzie
775c8cf756
Fix cross compilation issues for musl and aarch64 (#322)
* Enables tests for Musl and AArch64 (also rasbery pi, but leaves them disabled for now) and includes many fixes for the tests that were broken.

* Makes Musl cross compiler more like native (including making `ghci` and `runghc` work)

* Updates selection of enableShared

* Works around missing configFiles attribute issue

* Use ghc-extra-packages and compiler.nix-name to pick the right extra packages

* Leaves out --cross-compile on hsc2hs for musl

* Fixes haddock and hoogle for musl and disables them for other cross compilers

* Adds ghc 8.8.3

* Static link components (except libraries and `all`) on musl

* Use qemu for the arm cross compile `testWrapper`

* Add isCrossHost and isCrossTarget functions

* Fixes profiling on AArch64

* Disable split sections when compiling for windows (fixes GHC 8.8 cross compile to windows)

* Disable hoogle in cross compiler shells for now
2020-03-04 16:07:01 +13:00
Dennis Gosnell
ce65bf2c0e
Refactoring test/default.nix (#434) 2020-02-05 14:11:46 +13:00
Hamish Mackenzie
06ea6ab854
Update cleanSourceWith based on the latest nixpkgs (#401) 2020-01-15 13:56:18 +13:00
Hamish Mackenzie
8e52ad6f96
Automatically generate cache for stackage projects (#397)
You can add a `# nix-sha256` comment to stack.yaml if you
need it to work in restricted mode.
2020-01-15 10:34:14 +13:00
Rodney Lorrimar
d7d24fde4b
mkStackPkgSet: filter out $locals package (#393)
I found that if I added this to my stack.yaml:

    ghc-options:
      $locals: -ddump-to-file -ddump-hi

As specified in https://docs.haskellstack.org/en/stable/yaml_configuration/#ghc-options

Then my build would fail to evaluate with:

    error: The option `packages."\$locals".package.identifier.name' is used but not defined.

So to fix it, we filter out these special package globs in mkStackPkgSet.

This commit also adds a regression test for that configuration option.
2020-01-09 16:07:34 +10:00
Hamish Mackenzie
43fdc4dafd
Fix restricted eval mode issue with mkCacheFile (#392)
If the .nix files listed in the cache are full paths pointing at the
store we get errors like this in restricted eval mode:

error: access to path '/nix/store/...-cabal-simple.nix' is forbidden
in restricted mode
2020-01-07 19:07:25 +13:00
Kirill Elagin
a3b3e0e7eb test: Fails to use snapshot.yaml (#338) 2019-12-04 10:56:08 +08:00
Hamish Mackenzie
a9b01221dd
Replace collectRunComponents with haskellLib.check (#316)
This change replaces `passthru.run` and `collectRunComponents` with
`haskellLib.check` and `$pkg.checks`.

Uses `ln -s` instead of `cp` for DLLs to avoid duplicating them.

Adds features to help debugging eval issues with haskell.nix/release.nix.
The ifdInputLevel can be set to limit the jobs included to control
how much work is done at the eval stage when looking for issues.

Reduce CI times by excluding older ghc versions from haskellNixRoots
(GHC 8.6.3 and 8.6.4)

Sets `preferLocalBuild=false` on `callCabalProjectToNix` and `callStackToNix`
to ensure they are cached on hydra for both Linux and macOS.

Fix `data-dir:` when cleaning components.
`normalizeRelativeDir` adds a slash on to the end of `dataDir`.
Adding another one here results in `//` and files are left out by
mistake.
2019-11-29 23:28:47 +13:00
Hamish Mackenzie
79c2c631c3
Run the cross compiled tests on wine (#317)
Changes to the haskell.nix code to fix broken tests:

* Add missing `then` to `call-cabal-project-to-nix.nix`.

* Fix default `hsSourceDirs` so that `.` gets included for `.all` component if one of the components does not have a `hsSourceDir` set.

* Fix `haskellNixRoots` so it works when cross compiling to windows.

Improvements to the haskell.nix tests:

* Run haskell.nix tests of nixpkgs 19.03 and 19.09.

* Run haskell.nix tests cross compiled to Windows under Wine (when possible).

* Add nix used as IFD inputs as tests to ensure they are cached.

* Use `haskell-nix.cabal-install` instead of `nixpkgs.cabal-install` in tests.
2019-11-20 13:47:15 +13:00
Hamish Mackenzie
19248b5a15
Revert "Disable all the tests except the haskellNixRoots"
This reverts commit 2000983bb0.
2019-11-15 14:20:16 +13:00
Hamish Mackenzie
2000983bb0
Disable all the tests except the haskellNixRoots 2019-11-15 12:49:36 +13:00
Hamish Mackenzie
9b10945d98
Add passthru.run with wine support (#301)
This can be used to run tests, executables or benchmarks.
Instead of building drv build drv.run and the result should be the output of stdout.
2019-11-11 23:38:08 +13:00
Hamish Mackenzie
023863aeca
fix #269. Apply fix to mkStackPkgSet (#310)
First fix only fixed `haskell-nix.snapshot` by applying the fix also
to mkStackPkgSet stack.yaml based projects should work with lts 14
2019-11-09 17:01:04 +13:00
Hamish Mackenzie
c258167a50
Fix components.all for exe only packages (#295)
Currently the builder tries and fails to register a non existent
library component.
2019-11-05 19:49:12 +13:00
Hamish Mackenzie
612e983daa
Move all of haskell.nix into haskell-nix (#289)
Currently the haskell.nix overlays replace `haskell` and
`haskellPackages` in the root of pkgs with versions that have just a
fraction of what nixpkgs provides.
Although `haskell-nix.haskellPackages` could be used to provide a much
more complete set there are subtle differences that are likely to cause
issues (and nixpkgs has patches for many haskell packages).

To make it possible for haskell.nix and nixpkgs haskell to coexist,
we need to move the remaining parts of haskell.nix outside of the
`haskell-nix` attribute into `haskell-nix`.

```
  ghc                        -> haskell-nix.ghc
  cabal-install              -> haskell-nix.cabal-install
  haskell.compiler           -> haskell-nix.compiler
  haskell.packages           -> haskell-nix.packages
  bootstrap.haskell.compiler -> haskell-nix.bootstrap.compiler
  bootstrap.haskell.packages -> haskell-nix.bootstrap.packages
```

The following `haskellPackages` have been removed and instead the
nixpkgs versions will now be in `haskellPackages`.  User code should
be updated if the nixpkgs versions are not desired:

```
  haskellPackages.hoogle
    -> haskell-nix.haskellPackages.hoogle.components.exes.hoogle
  haskellPackages.happy
    -> haskell-nix.haskellPackages.happy.components.exes.happy
  haskellPackages.alex
    -> haskell-nix.haskellPackages.alex.components.exes.alex;
```

Guard code has been added to overlays/defaults.nix to prevent us
inadvertently depending on the nixpkgs `haskell` or `haskellPackages`
inside the haskell.nix code base.
2019-11-05 12:50:50 +13:00
Hamish Mackenzie
1640c35ed2
Add names to tests that have an index-state (#288) 2019-11-04 11:29:34 +13:00
Rodney Lorrimar
686fa37877
shellFor: Fix bug where packages were not excluded from the env (#285)
* tests: Add regression test for shellFor build-tools bug

* tests: Fix regen script

* Regenerate nix expressions for tests

* tests: ghc 8.4.4 -> 8.6.5

* shellFor: Fix bug where packages were not excluded from the env

If a multi-package project had an build-tools dependency between
packages, they would not be correctly removed from the shell's inputs.
2019-11-01 20:08:31 +10:00
Hamish Mackenzie
a5f9f45922
Test setup deps issue depending on ghc and Cabal (#278)
Projects like asterius and leksah that include dependencies on ghc. Sometimes the are needed by a setup.hs file (as is the case for the binaryen package used by asterius). The recent overlays merge broke this and #279 was needed to allow reinstallableLibGhc to be used to fix it. This PR adds a test case to make sure it is not inadvertently broken again. Also gives us a place to try out better ways to make this type of package work.* Test setup deps issue depending on ghc and Cabal

* Uses nixpkgs 19.09 + macOS wine fix on hydra/buildkite

* Enables macOS hydra tests

* disables musl tests for now

* Adds haskellNixRoots to tests

* Uses reinstallableLibGhc to fix the new setup-depends test

* Adds meta.platforms to haskell.nix components

* Set meta.platforms on cabal-install

* Builds some maintainer-scripts on hydra

* Pins hackage index-state for ghc-extra-packages
2019-11-01 12:34:10 +13:00
Moritz Angermann
5884ab02be
Overlays (#261)
The Overlays branch

This is a major reorganization in how haskell.nix is used, and marks our 1.0 release.
The branch doesn't build due to numerous issues that we believe to be with the CI
and not this branch. We expect only very minor adjustments prior to calling this the
official 1.0 release.

* Move iohk-nix patches into haskell.nix

This moves the customizations we did in iohk-nix
into haskell.nix via overlays and config.

Add bootPkgs logic

this moves the nuking of bootPkgs where it belongs. This should eventually still be
removed and replaced by a proper solution, that doesn't require the nuking of
bootPkgs.

Allow us to bootstrap binary ghcs and a cabal-install

With this we can do the following:

```
$ nix repl compiler/old-ghc-nix
nix-repl> :b (let pkgs = import <nixpkgs> {}; in with import ./. {}; import ./compiler/bootstrap/cabal-install.nix { inherit (pkgs) fetchurl stdenv zlib; inherit hackage
; ghc = ghc844; src = pkgs.fetchurl { url = "https://github.com/haskell/cabal/archive/Cabal-v3.0.0.0-rc3.tar.gz"; sha256 = "1zl2mgg8307ykq3v8nmafc6zdhhj1cw7w8ffpap16dsm6
5lbnx33"; }; })
```

which wile it may look daunting, will allow us to bootstrap a
cabal-install with a ghc. From that point onwards, we should be able to
build any hackage package via haskell.nix.

Pass through cabal-install version

Better threading of arguments.

Add bootstrap overlay

Allow alex + happy to be built

This still has a wart: we need nix-tools, and for that we use the ghc865 from
nixpkgs. Ideally we'd build nix-tools against a specific ghc, but then
we'd need a build expression for that.

Make ghcjs work

Building something like this:
```
 nix build '(with import ./. { nixpkgs = ../nixpkgs; nixpkgsArgs = { crossSystem = { config = "js-unknown-ghcjs"; }; }; }; (haskell-nix.hackage-package { name = "hello"; version = "1.0.0.2"; })).components.exes.hello'
```
will now work. Assuming `nixpkgs` has been appropriately patched to support the `js-unknown-ghcjs` triple.

Also: this will need an additional `Cabal` patch, to make `Cabal` understand what it needs to do with: `dist/build/hello/hello: copyFile: does not exist (No such file or directory)`

It needs to learn that `dist/build/hello/hello.jsexe` is what it wants to copy and that that is a directory.  Luckily we do have some code in Cabal already that does this for `.exe` on windows.

Build `js-unknown-ghcjs` packages with haskell.nix

Using the following expression:
```
with import ./. { nixpkgs = ../nixpkgs; nixpkgsArgs = { crossSystem = { config = "js-unknown-ghcjs"; }; }; };
let Cabal = buildPackages.haskell-nix.hackage-package {
    name = "Cabal"; version = "2.4.1.0";
    modules = [
        { packages.Cabal.patches = [ ./Cabal-install-folder.diff ]; }
    ];
}; in
(haskell-nix.hackage-package {
    name = "hello"; version = "1.0.0.2";
    modules = [
    	    ({config, ... }:{ packages.hello.package.setup-depends = [ Cabal ]; })
    ];}).components.exes.hello
```
in a `test.nix` file. And running
```
nix build -f ./test.nix
```
on it, will produce
```
./result
├── bin
│   └── hello.jsexe
│       ├── all.js
│       ├── all.js.externs
│       ├── index.html
│       ├── lib.js
│       ├── manifest.webapp
│       ├── out.frefs.js
│       ├── out.frefs.json
│       ├── out.js
│       ├── out.stats
│       ├── rts.js
│       └── runmain.js
└── share
    └── doc
        └── x86_64-linux-ghc-8.6.5
            └── hello-1.0.0.2
                └── LICENSE

6 directories, 12 files
```
2019-10-21 20:07:58 +08:00
Hamish Mackenzie
94f4679618 Add setup-depends to shellFor (#259)
* Add failing test for shellFor with setup-depends

* Add setup-depends to shellFor
2019-10-13 20:01:25 +08:00
Rodney Lorrimar
e4a572b034 tests: Add test case with fully static build of pandoc (#256)
* tests: Add test case with fully static build of pandoc

It uses the musl libc. To try it, use:

   nix-build test/default.nix \
     --arg crossSystem '(import <nixpkgs/lib>).systems.examples.musl64' \
     -A fully-static.pandoc-gmp

This includes the GMP library (linked statically). To build one without GMP:

   nix-build test/default.nix \
     --arg crossSystem '(import <nixpkgs/lib>).systems.examples.musl64' \
     -A fully-static.pandoc-integer-simple

Unfortunately, the latter doesn't work (it says "ghc" command not found).

* fixup! tests: Add test case with fully static build of pandoc
2019-10-13 19:44:24 +08:00
Rodney Lorrimar
b47a6bb1a2 shellFor: Use CABAL_CONFIG only with exactDeps = true (#255)
* shellFor: Use CABAL_CONFIG only with exactDeps = true

* shells: Put write-ghc-environment-files: never in cabal.config
2019-10-13 19:43:17 +08:00
Hamish Mackenzie
df51ce67b2 Fix conflicting flag definitions #254 (#258) 2019-10-13 19:42:26 +08:00
Hamish Mackenzie
147cf21341 Add ghcOptions on packages (#257)
* Add `ghcOptions` with example and docs
2019-10-13 14:19:59 +08:00
Hamish Mackenzie
f648a6585e
Fix project flags (#253)
Project flags (from stack.yaml and plan.json) are exported in a modules attribute by stack-to-nix and plan-to-nix, but are not currently used. This change updates `mkStackPkgSet` and `mkCabalProjectPkgSet` so that the modules attribute is used (if present) and includes tests to check they are.

This commit makes `stdenv.lib.mkOverride` necessary for setting flags in `mkCabalProjectPkgSet` modules, however it also means that you can set them in `cabal.project` instead (see #254).
2019-10-11 17:19:31 +13:00
Rodney Lorrimar
53c0498ec4 Add tests for benchmarks and hspec-discover (#181)
* tests: Add benchmarks and hspec-discover to cabal-22 test
2019-10-01 10:01:49 +08:00
Hamish Mackenzie
fdcb2da2a8
Add default for shellFor packages (#242)
Uses the `isLocal` for property set by `plan-to-nix` and `stack-to-nix`
on packages where the source is referenced in the `cabal.project`
or `stack.yaml` file.
2019-10-01 11:10:11 +13:00
Hamish Mackenzie
6f566a6d5d Filter components using buildable flag (#240)
* Filter components using buildable flag

* Bump nix-tools

* Include components without buildable flag

* Test buildable components are filtered correctly
2019-09-26 13:47:03 +08:00
Hamish Mackenzie
0366537651
Use new cleanSourceWith to filter component source (#209)
When trying to build haskell packages from local source using nix (
source code you have used `git clone` to retrieve and have modified)
it is often annoying that small changes can trigger a lot of components
to be built unnecessarily.  For instance if you change the code for
a test in a package it will often trigger builds of the library and
all its dependencies.

To avoid this problem we could manually set up `cleanSourceWith` calls
in nix to filter out files that are not need, but to do this every
component would be difficult and error prone.

This change automates this process using the information in the
packages `.cabal` file to filter the source of each component
automatically  when using `callCabalPlanToNix` (we should be able to
add it to `callStackToNix` too).

## How does it work?

* A new version on `cleanSourceWith` is used that adds a `subDir`
  argument.  It allows descending into a subdirectory to compose with
  cleaning the source.

* A new `--full` argument is passed to `plan-to-nix` by
  `callCabalPlanToNix` telling it to output extra information about
  the cabal packages in the plan (directories and filenames).

* `callCabalProjectToNix` uses the new `subDir` argument of
  `cleanSourceWith` when setting the packageSrc.

* `comp-builder.nix` and `setup-builder.nix` use a new
  `cleanCabalComponent` function to further clean the packageSrc
  using the new information in the `package` and `component`.

## hpack wildcards
`package.yaml` files can include wild cards and these are not handled
yet (they do not map to anything in the `.cabal` file and so are lost).
They will need to be added manually to the appropriate field
(eg. `packages.xyz.package.dataFiles`)

If your hpack file does not contain wildcards or if you are willing to
add the wildcards entries manually in nix it is still useful to
enable cleaning on hpack.

For example:

```
packages.ghc-toolkit.package.cleanHpack = true;
packages.ghc-toolkit.components.library.extraSrcFiles = [
  "genapply/**/**"
  "boot-libs/**/**"
  "ghc-libdir/**/**"
  ];
```
2019-08-26 12:27:47 +02:00
Hamish Mackenzie
dedb91e725 Add sha256 to source-repository-packages for restricted eval mode (#170)
* Add sha256 to source-repository-packages for hydra

This allows source-repository-package to be downloaded and replaced in
with `package: /nix/store/...` by callCabalPackageToNix so that `cabal`
and `plan-to-nix` do not need to download them.
2019-07-16 15:13:31 +08:00
Hamish Mackenzie
3d02d751f7 index-state from cabal.project by default (#169)
* index-state from cabal.project by default

Change callCabalProjectToNix to use the `index-state` specified in the
`cabal.project` file by default.
2019-07-16 13:28:20 +08:00
Rodney Lorrimar
f81a37eda4
Make sure hoogle is available in ghcWithHoogle (#196) 2019-07-16 10:17:49 +10:00
Hamish Mackenzie
6be886644f Cross comp. & callCabalProjectToNix/callStackToNix (#202)
* Cross comp. & callCabalProjectToNix/callStackToNix

This fixes an issue where the these IFD functions fail because they try
to build and run a cross compiled nix-tools on the build system.

It may also help if there are cross compilation issues with
setup-depends (they also access the buildPackages).

* Align stack-to-nix ifd with plan-to-nix ifd

This also fixes the restricted/pure eval issues with callStackToNix.

I've also made the tests run in restricted mode to catch these kinds of
issues.
2019-07-13 22:13:15 +08:00
Hamish Mackenzie
efab350c0f Fix setup-depends dependencies (#124)
* Use setup-depends from nix-tools output

* Use `-package ghc` when building Setup.hs

* Include --package-db when building Setup

* Fix for tests on case insensitive FS (macOS)

The file `conduit.hs` looks like `Conduit.hs` to ghc and it uses it
by mistake for `import Conduit`.
2019-07-12 21:48:13 +08:00
Rodney Lorrimar
3bb58214f1
Add cross-compiled musl libc builds of tests (#157)
* Add cross-compiled musl libc builds of tests

* Bump nixpkgs to latest 19.03

* tests: Update stack-simple LTS

* docs: Add note about tests

* Regenerate tests

* haskellPackages: lts-13.18 -> lts-13.26

This contains GHC 8.6.5 which can successfully build under
pkgsCross.musl64, whereas GHC 8.6.4 could not.

* Bump nix-tools

* Regenerate nix-tools -- changed compiler from GHC 8.6.4 -> GHC 8.6.5.
2019-07-02 18:25:40 +10:00
Rodney Lorrimar
0663a9cd03
Add ghcWithPackages and ghcWithHoogle to hsPkgs (#150)
* Add ghcWithPackages and ghcWithHoogle to hsPkgs

* tests: Add ghcWithHoogle test

* Update changelog
2019-06-21 22:40:08 +10:00
Rodney Lorrimar
097852792d Fix daily Hackage update (#177)
Hackage hasn't been updated for a week because of a build error.
2019-06-15 07:16:52 +02:00
Moritz Angermann
ea6f125ca4
Adds profiling support (#162)
* Adds profiling support

* Drop debug info
2019-06-04 11:28:14 +08:00
Rodney Lorrimar
caf1f61267 Use release-lib.nix for Hydra (#161)
* Use release-lib.nix for Hydra jobset

* Buildkite: Add a script to check Hydra evaluation
2019-06-03 11:26:28 +08:00
Rodney Lorrimar
8512832c32
Fix evaluation on Hydra (#153)
* Fix eval on Hydra

* Bump nixpkgs to latest 19.03

* Change unit tests into a derivation

Prevents Hydra from complaining about them.

* Add unpacked source pins to Hydra jobset

So that they are cached.
2019-06-02 07:21:21 +10:00
Jean-Baptiste Giraudeau
3fb220334f
Fix name of shellFor derivation for a single package. (#155) 2019-05-30 22:46:39 +02:00
Rodney Lorrimar
2cbfbac176
tests: Add a test case for the shellFor hoogle index 2019-05-28 10:53:05 +10:00
Rodney Lorrimar
e6b07b482b
tests: Generated nix for cabal new-build project 2019-05-28 09:35:00 +10:00
Rodney Lorrimar
dd1baeb287
tests: Add a cabal new-build project 2019-05-28 09:35:00 +10:00
Rodney Lorrimar
b060ea5576
tests: Add tests for the shellFor function 2019-05-28 09:34:58 +10:00
Rodney Lorrimar
37a1b7ac2b Add snapshots attribute (#151)
* Add a snapshots attribute

These contain package sets for all of the Stackage snapshots.

Relates to #22

* tests: Add stackage snapshots test

* Add a stackage fix for "No attribute: hsc2hs"

* Add comment to snapshots.nix and fix version comparison function
2019-05-27 16:05:42 +08:00
Moritz Angermann
dfefba5be7
Call cabal project to nix (now with hackage-truncate) (#135)
* This adds more logic for the cabalProjectToNix idf

That is, something like this:

```
nix-build --expr 'with import ./. {}; callCabalProjectToNix { index-state = "2019-04-30T00:00:00Z"; src = /some/path; }'
```

should produce something that can be build with mkCabalProjectPkgSet.

* Make sure the hackageTarball's store path doesn't change

previously the fetchurl would produce a different store path each
and every time as hackage's index is a moving target.  With this
impurity setup, we can ignore this.

* Fix test

* Proper name

* Re-enable test

* Allow to parameterize over the `system` for the test default.nix

* Copy instead of link for ifds.

This makes me really sad.
2019-05-21 20:05:03 +08:00
Rodney Lorrimar
fa6327eb24 Try to fix tests (#138)
* tests: Regenerate nix

* tests: Use mkCabalProjectPkgSet

* tests: Disable callCabalProjectToNix test
2019-05-20 21:29:02 +08:00
Moritz Angermann
52280f2324
[ifd] cabalProjectToNix (#122) 2019-05-18 22:00:28 +08:00
Moritz Angermann
39e77c4889
Fix tests (#133)
With the nixpkgs bump (#131) there is no ghc-8.6.3 anymore.
2019-05-18 17:42:23 +08:00
Edward Amsden
390771849a Add callStackToNix function (#116)
Allow building a stack project directly without needing to generate the scaffolding.

```nix
{ mkStackPkgSet, callStackToNix, ... }:
let
  pkgSet = mkStackPkgSet {
    stack-pkgs = callStackToNix { src = ./.; };
    pkg-def-extras = [];
    modules = [];
  };
in pkgSet.config.hsPkgs
```
2019-05-17 15:49:40 +08:00
Moritz Angermann
072debc663 Fixup tests 2019-05-17 13:02:40 +08:00
Rodney Lorrimar
00c941466d
doc: Update test/README.md 2019-05-08 09:50:31 +10:00
Rodney Lorrimar
515c88e5ab Add script for regenerating nix in tests
And update it for new versions of plan-to-nix.
2019-05-07 15:31:35 +08:00
Rodney Lorrimar
76a31e34d8 Regenerate nix for test/cabal-sublib
To avoid transformers-0.5.0.0 problem
2019-05-07 15:31:35 +08:00
Rodney Lorrimar
d8c13de415 Regenerate nix for test/with-packages to avoid transformers-0.5.0.0 problem 2019-05-07 15:31:35 +08:00