Commit Graph

300 Commits

Author SHA1 Message Date
phaer
c67109b0f1 docs: add devShell hook 2024-06-13 13:37:45 +02:00
phaer
5c6e0e5e9a docs: add mkdocs-awesome-pages-plugin 2024-06-13 13:06:22 +02:00
phaer
6ff41abd61 docs: fix devshell 2024-06-13 11:54:11 +02:00
phaer
bb4b27886f add render_options hook 2024-06-11 19:08:00 +02:00
phaer
42ff2e8d4d move dreamDocs out of docs dir
to reduce unecessary rebuilds
2024-06-11 19:08:00 +02:00
phaer
7cb146701a add mkdocs-based docs site 2024-06-11 19:05:26 +02:00
purepani
478b26af7a
pdm: Adds editables to pdm 2024-06-11 11:12:03 -05:00
purepani
0a17d6a101
pip: Factors python editables into separate module 2024-06-11 11:11:20 -05:00
Paul Haerle
d872284158
Merge branch 'main' into python-pyproject-format 2024-05-23 11:48:48 +02:00
Paul Haerle
6a169a73bd
Python DevShells (for pip) (#951)
* fetchPipMetadata: set meta.mainProgram

* fetchPipMetadata: deduplicate test fixtures

* fetchPipMetadata: add is_direct to lock file

* devShell: proof of concept

* python-local-development: add editables example

* devshell: simplify editables interface

* buildPythonPackage: remove editable option again

* devshell: use findRoot for editables

* devshell: add support for .whl sources

* devshell: avoid polluting the shells environment

* devshell: make editable.nix flatter

because there's only 1 attr left now

* devshell: fix pyEnv by filtering editables

from the environment.

* devshell: skip existing editables

* devshell: remove patched pyEnv, rewrite sys.path

* devshell: add special case for root package

* python-local-development: improve editable example

* python-local-development: filter source

* devshell: reset site_dir on each load

* pip: don't ignoreCollisions in pyEnv

* pip: default to no sitecustomize.py...

and teach the devshell how to load it

* pip: rewrite editable in python

* editable: use shutil.copytree

* editable: refactor into functions

* editables: add suport for console_scripts

* improve pip editables interface

- change editables interface to use bool or path
- expose public.shellHook for composition
- use shellHook by default in public.devShell
- extend example to include a script

* improve edtiable support:

- remove dependency on root package build
- always set root package as editable by default
- compute dit-info dir by calling a packages build backend or extracting directly from the wheel
- only pass required sources to editables shell hook
- add build inputs of all editables to teh dev shell
- move editables state to .dream2nix/python to remove likelyhood of collision with other ecosystems state
- delete editables state if editables configuration changed

* refine python editable support:

- never copy dependency sources
- force user to specify a local path

---------

Co-authored-by: DavHau <hsngrmpf+github@gmail.com>
2024-05-17 01:00:09 +02:00
Jairo Llopis
4cd8cbdb20 feat(buildPythonPackage): support new pyproject format
https://github.com/NixOS/nixpkgs/pull/271597 implemented a new format for python packages that is starting to get traction in nixpkgs/unstable (soon to become NixOS 24.05).

That is a progress towards https://github.com/NixOS/nixpkgs/issues/272178, which would be a massive improvement for dream2nix once completed.

This first step just makes sure that `buildPythonPackage` supports the new arguments properly. Without this change, many dream2nix python derivations fail to build on nixos-unstable right now.

Probably other python auto-builders should leverage these new options when available for the d2n user. That can be added later.

@moduon MT-1075
2024-05-13 09:40:15 +01:00
Peter Ferenczy
32aa32d397 fmt 2024-04-28 21:46:53 +07:00
Peter Ferenczy
62c6361657 mkDefault 2024-04-28 21:46:53 +07:00
Peter Ferenczy
3e4cd42a9f add rsync to nodejs-devshell-v3 2024-04-28 21:46:53 +07:00
purepani
babb6e3b69 python/pdm: Changes deps.python3 refs to deps.python 2024-04-28 13:18:00 +07:00
phaer
c5cfd416ab core.lock: Add missing coreutils dependency...
otherwise we implictly assume realpath to already
be in $PATH which isn't always the case, i.e. on older
macOS releases.
2024-04-15 19:30:05 +07:00
phaer
d3f8dbfc79 fix typo 2024-04-15 19:30:05 +07:00
shivaraj-bh
49f14262e3 WIP-python-pdm: add public.pyEnv; reuse public.pyEnv in public.devShell 2024-04-08 18:24:01 +07:00
DavHau
2fc4ef33cc docs: overhaul reference documentation pipeline
- use nixosOptionsDoc commonMark output
- simplify a lot of things
- redesign towards a future generic standalon nixosOptionsWebsite function
2024-04-05 19:42:14 +07:00
lbodor
26a6c3a340 WIP-haskell-cabal: fetch correct cabal revisions
Fetch cabal revisions from `https://hackage.haskell.org/package/{name}-{version}/revisions/`, rather
than `https://hackage.haskell.org/package/{name}/revisions/`, which fetches revisions for the latest
package version, not the version pinned in `./dist-newstyle/cache/plan.json`.
2024-03-29 03:56:14 +00:00
purepani
4710341e67 python/pdm: Fixes order of build inputs in devShell so that the path to
path to the interpreter is correct.
2024-03-29 02:50:26 +00:00
DavHau
6ddb9a7f5e override/python: fix devShells for nvidia packages 2024-03-26 05:53:26 +00:00
DavHau
306e5bff07 pythonEnv: init
Add experimental one-liner command to create python environments.

```command
$ nix run github:nix-community/dream2nix#__pythonEnv requests whisper-ctranslate2
...

$ python -c "import requests; print('imported requests')"
imported requests

$ whisper-ctranslate2 --version
whisper-ctranslate2 0.4.1
```
2024-03-26 05:03:45 +00:00
Robert Hensing
d623e6ec21 mkDerivation: fix typo in checkFlags 2024-03-22 21:02:23 +07:00
DavHau
f017de2de4 mkDerivation: stop depending on all core modules
mkDerivation should be minimal, but currently it always pulls in modules like `lock` and `paths` etc. despite those not being needed for many instances, like when mkDerivation is imported by a dependency.

Modules like pip which do need some of these core modules, now need to directly depend on them.

This will remove a bunch of unneeded options which currently pollute the manual.

The remaining core dependencies of mkDerivation are now: deps, env, ui
2024-03-18 11:15:30 +00:00
DavHau
f15cc37662 mkDerivation/builtins-derivation: refactor: implementation.nix -> default.nix 2024-03-17 22:55:10 +07:00
DavHau
6bbac51494 python/pdm: hide groups as implementation detail
The most important feature of the groups module was the overriding which is a separate module now. Therefore removing the fact that groups are used from the docs to decrease the API surface to maintain.

Instead render the override and overrideAll options in the manual of pdm correctly by setting `overrideType`.
2024-03-17 15:48:07 +00:00
DavHau
41a1973ed3 groups: remove options overrides and overrideAll
...since those are now managed by the dedicated overrides module
2024-03-17 14:37:33 +00:00
DavHau
86e4b94689 docs/overrides: explain how to find options 2024-03-17 14:34:12 +00:00
DavHau
b6e6d36f9d docs: init overrides 2024-03-17 09:27:24 +00:00
DavHau
b8d00b7c98 docs: update module system intro 2024-03-17 06:04:46 +00:00
DavHau
b727a07775 override: unify overriding across modules
Add options `overrides.${name}` and `overrideAll` to all language modules that manage a dependency tree.

This is done in a backward compatible fashion. Old options for overriding dependencies continue working, though those are not displayed in the manual anymore.

For the following modules the manual now suggest using the new unified override options:
- for pip: use `pip.overrides` and `pip.overrideAll` instead of  `pip.drvs`
- for nodejs-granular-v3: use `nodejs-granular-v3.overrides` and `nodejs-granular-v3.overrideAll` instead of `nodejs-granular-v3.deps`
- `php-granular`: use `php-granular.override` and `php-granular.overrideAll` instead of `php-granular.deps`
2024-03-13 14:25:41 +07:00
DavHau
63ab24d893 docs: refine public/experimental
Move some barely used modules to the experimental section to reduce surface.
Instead of the separate dev-shell module the nodejs package should have a devShell subattribute.
2024-03-12 11:25:12 +00:00
DavHau
45d941d49f docs: don't strip foreign module options
previously the options of other imported modules were not shown which gave an incomplete picture. For example the docs page for the `pip` module did not show the options prefixed with `mkDerivation` which are accepted and needed by the pip module.
It's better to have everything on one page for each module.
2024-03-12 11:05:59 +00:00
DavHau
ba4be52cdf ci: build website only on x86_64-linux to reduce build time
The website builds really slow on other platforms due to the buildbot remote builder setup.
2024-03-12 10:08:01 +00:00
DavHau
211528826c cleanup core options
- don't import eval-cache and flags modules by default (those have never been used so far)
- mark options of package-func as internal. Those are for maintainers only and don't need to be rendered by default
2024-03-12 09:41:59 +00:00
DavHau
bb13da673a checks.nix-unit: use nix-unit from nixpkgs 2024-03-12 07:09:04 +00:00
DavHau
63c9f51a43 pdm: add test for jupyterlab 2024-03-11 14:24:27 +00:00
DavHau
deb4c98014 docs: fix links to examples 2024-03-11 10:41:05 +00:00
DavHau
42838c5909 examples: cleanup and rename to simplify
- name example more consistently, eg. `{language}-packaging-{feature}` or `{language}-local-development-{feature}`
- move some examples to the modules integration tests directory instead as their purpose was mainly testing and they weren't good example
- module owned checks: import via flake if available
2024-03-09 04:45:59 +00:00
DavHau
3b9e0780a0 docs: separate public and internal modules 2024-03-07 08:51:00 +00:00
DavHau
5f6c966ab3 update-flake: fix to work with multiple directories once 2024-03-07 05:22:52 +00:00
DavHau
e8f018187c fix collision issue with deps.bash 2024-02-28 07:08:39 +00:00
DavHau
887022b8c1 update nixpkgs:
- downgrade mitmproxy to avoid breakage
- add test for pip lock script
- fix bug in flake compat implementation (https://github.com/nix-community/flake-compat/pull/4#issuecomment-1950301370)
- small fix in docs for lock and eval-cache
- fix lock refresh script: don't swallow errors
2024-02-24 13:47:45 +00:00
DavHau
3308e05808 pdm: add rudimentary devShell
Also fix lock script to not swallow error messages
2024-02-20 11:53:54 +00:00
DavHau
5a5ae25b04 fetching: set shallow=true for all fetchGit calls
This should lead to better performance
Also set allRefs=true in some places, as this reduces the risk of running into issues with older nix versions
2024-02-18 14:08:51 +07:00
Johannes Kirschbauer
3065f50344 nodejs: handle more npm dependency types 2024-02-16 09:44:32 +00:00
Johannes Kirschbauer
eed2342f7f nodejs: move installation into derivation module 2024-02-16 03:59:35 +00:00
Johannes Kirschbauer
10b523238c nodejs: move outputs in seperate module files 2024-02-15 06:42:10 +00:00
Arnout Engelen
367e7fcc98 Getting started docs
I always forget how to get going with dream2nix, and while this
page doesn't contain much more than you'd already find in the
examples, I do think it's a good starting point for allowing
people to start more confidently.
2024-02-09 20:33:18 +07:00
DavHau
16f2c654c6 pdm: stop using config.deps.python3 in favor of config.deps.python 2024-01-24 08:09:56 +00:00
DavHau
0a30902e27 pdm: fix various bugs
- default to correct version of setuptools from lock file
- better default for python3 set in dream2nix.core.paths
2024-01-24 07:29:32 +00:00
Jairo Llopis
233c4d8106 fix(nodejs-package-json-v3): remove duplicate dependency 2024-01-23 20:32:36 +07:00
Vincent Vanlaer
eb3b7028d5 pip: allow git+ssh dependencies 2024-01-23 20:31:31 +07:00
Herman Fries
774019d486 lock: Extend environment instead of replacing it
This allows running impure commands, that rely on user's configuration. E.g. to access a private gitlab project.
2024-01-20 02:05:05 +00:00
Johannes Kirschbauer
bf07b8fb37 add filesystem test 2024-01-19 12:02:18 +00:00
Johannes Kirschbauer
df26fb319f simplify graph methods & add more tests for fileSystem 2024-01-19 12:02:18 +00:00
DavHau
4f2a64a66f nodejs-v3-builder: ensure top-level config is exported 2024-01-18 13:22:00 +00:00
Johannes Kirschbauer
a27d3894ae
Feat/group pkgs versions (#757)
* add derived graph

* rework graph sanitizing based on list keys

* figure out details for builder

* format stuff!

* try get groups working

* nodejs: fixups for new v3 builder

Co-authored-by: Johannes Kirschbauer <hsjobeki@gmail.com>

---------

Co-authored-by: DavHau <hsngrmpf+github@gmail.com>
2024-01-18 19:56:00 +07:00
Johannes Kirschbauer
57fd52adcd run install scripts: init
add option: 'trustedDeps'. Only install scripts of explizit added dependencies are run.
In the future there may be wildcards to allow running all install scripts
2024-01-18 09:15:06 +00:00
f
0af1c36d17 change empty string to empty array 2024-01-17 12:56:20 +07:00
phaer
3fc4a2ad97 pdm: only default to given python3 in overrideAll. 2024-01-17 12:50:16 +07:00
Johannes Kirschbauer
8300ad2d78 add custom unpackPhase from the original nodejs builder
- [x] examples still build
2024-01-12 17:03:39 +07:00
DavHau
00f1223642 pdm: default to setuptools as build system
This is the upstream default, so it doesn't hut to just include it everywhere
2024-01-08 15:45:18 +00:00
DavHau
ca706acbad pdm: lock fully, remove --refresh
--refresh only works well with already existing lock files
2024-01-08 22:41:18 +07:00
DavHau
0fb528d452 pdm: refresh pdm.lock on 'nix run .#{package}.lock' 2024-01-07 17:40:55 +00:00
DavHau
28f44827e4 fix(pip): prevent conflict if setuptools is runtime dependency 2024-01-07 13:33:48 +00:00
DavHau
935df57a46 pip: add test can-build-setuptools
to ensure that a setuptools dependency doesn't raise a collision
2024-01-07 13:00:47 +00:00
DavHau
a6837ba21e pip: switch to format = "pyproject"
This should resolve collision issues with the setuptools that gets propagated by setuptoolsBuildHook
2024-01-07 12:37:25 +00:00
DavHau
0f2d06b43e examples: add readme for packages examples 2024-01-07 06:28:34 +00:00
DavHau
62f9486535 overrides: add pillow, psycopg2 2024-01-06 09:49:31 +00:00
Arnout Engelen
1741d0785e nodejs-package-lock-v3: basic support for git+ urls 2024-01-03 10:09:30 +07:00
Arnout Engelen
2998fff950 nodejs-package-json(-v3): add git to the npm lock fetch script path 2024-01-03 10:09:30 +07:00
DavHau
262198033e python/pdm: patch binaries + set some better defaults 2023-12-13 08:50:30 +00:00
DavHau
1d63a866d7 python/pdm: use fetchFromLegacy
...until https://github.com/nix-community/pyproject.nix/pull/41 is resolved
2023-12-12 12:28:47 +07:00
DavHau
6aa5a29244 chore: remove more unnecessary flake outputs
...to make buildbot eval quicker
2023-12-10 06:10:36 +00:00
DavHau
95fa909f84 docs: remove spam from flake-outputs
...instead use internal options to pass the module docs pages to the renderer
2023-12-10 06:03:22 +00:00
DavHau
3f6c7cbd80 python/pdm: several improvements
- infer build-systems from pyproject.toml
- add more generic getClosure lib function to get the closure of any given dependency + extra
- resolve extras correctly for each computed closure
- overhaul some of the existing tests
2023-12-10 05:47:44 +00:00
DavHau
916bca75a1 nixpkgs-overrides: map env field correctly
Now that mkDerivation also has an `env` flag, we need to make sure to not forward this to mkDerivation, as we have our own env option.
2023-12-09 11:13:33 +00:00
DavHau
7292ee5455 python/pdm: improve implementation and UI
- implement per dependency sourceSelector option
- update pyproject.nix
- update nix-unit
2023-12-04 07:46:08 +00:00
DavHau
9ce00f86e9 update pyproject-nix 2023-12-03 10:35:52 +00:00
DavHau
4b155e6a5d docs: render docs also for core modules 2023-12-03 08:53:50 +00:00
DavHau
2f344133c3 groups: rename to WIP-groups
to make the status of this module more clear
2023-12-03 07:41:58 +00:00
DavHau
c25e88974f nodejs-devshell-v3: init 2023-11-17 07:35:09 +00:00
DavHau
db35339b1e test: add per module testing infra
So far, /examples was the only place with packages that are built in the CI pipeline.
This change allows us to add package build tests to individual modules at the location where the module is defined. This has several benefits:

- We are not forced to add unnecessary examples in order to maintain test coverage
- When modules get removed, their tests will be removed alongside
- It allows us to keep old modules around and assure they keep working without having to keep outdated examples in /examples
2023-11-17 07:27:54 +00:00
Yusuf Bera Ertan
3f1cbc8e93
feat(rust): update & support crane version v0.15.0, fix some issues (#788) 2023-11-10 17:11:30 +10:00
Yusuf Bera Ertan
f1a9f36bf7
fix(rust-cargo-lock): handle workspace = true for license, description and homepage package attributes (#787) 2023-11-09 00:11:56 +10:00
DavHau
7cb0fded82 nodejs: fix backward compat for nodejs-package-json
- revert recent changes on nodejs-package-json
- instead introduce nodejs-package-json-v3

This way, existing users won't face breakages in the api
2023-11-08 07:34:15 +00:00
DavHau
24e81d5ce8 examples: rename repo examples 2023-11-07 19:08:27 +07:00
DavHau
3030cd5877 examples/python-pdm: migrate to ./packages 2023-11-07 19:08:27 +07:00
DavHau
bce7ab631d nodejs-package-json: use v3 lock file 2023-11-07 18:45:35 +07:00
DavHau
9f507c9341 docs: improve docs for groups 2023-11-07 16:34:00 +07:00
DavHau
3b42e5a8e4 docs: fix missing options for modules
- render all top-level options declared by a module
2023-11-07 16:34:00 +07:00
DavHau
edda831f76 groups: update readme 2023-11-07 15:39:25 +07:00
DavHau
581a7cff0b flake: separate top-level from dev-flake fully 2023-11-05 03:42:18 +00:00
DavHau
d8e4b477e3 refactor: move dev related things to /dev-flake
The goal is to reduce the flake inputs visible to the user
Still, the top-level flake re-exposes the outputs of the dev-flake, but
without exposing its inputs. This means a devShell is still available in
the top-level, for example.

This also removes the /modules/flake.nix. Its original purpose was
separating the modules inputs from the development inputs, but this is
now done the opposite way around by moving the dev inputs to
/dev-flake/flake.nix.
2023-11-05 03:42:18 +00:00
DavHau
1f680838ab lib: move to /lib from /modules/flake-parts/lib 2023-11-05 03:23:11 +00:00
DavHau
0ed0a765d6 pdm: update pyproject-nix dependency 2023-11-04 06:36:34 +00:00
adisbladis
2c8f520108 pdm: Use fetchFromPypi from pyproject.nix 2023-11-04 06:36:34 +00:00
adisbladis
e28ae15e36 pdm: Remove universal wheel file name test
It was broken by changes in pyproject.nix and doesn't seem to be in use anyway.
2023-11-04 06:36:34 +00:00