Commit Graph

245 Commits

Author SHA1 Message Date
purepani
cbe72c89ba
python-editables: Exports environment variables for python whether
.dream2nix needs updating or not (editables.py)
2024-06-23 15:20:56 -05:00
purepani
478b26af7a
pdm: Adds editables to pdm 2024-06-11 11:12:03 -05:00
purepani
0a17d6a101
pip: Factors python editables into separate module 2024-06-11 11:11:20 -05:00
Paul Haerle
d872284158
Merge branch 'main' into python-pyproject-format 2024-05-23 11:48:48 +02:00
Paul Haerle
6a169a73bd
Python DevShells (for pip) (#951)
* fetchPipMetadata: set meta.mainProgram

* fetchPipMetadata: deduplicate test fixtures

* fetchPipMetadata: add is_direct to lock file

* devShell: proof of concept

* python-local-development: add editables example

* devshell: simplify editables interface

* buildPythonPackage: remove editable option again

* devshell: use findRoot for editables

* devshell: add support for .whl sources

* devshell: avoid polluting the shells environment

* devshell: make editable.nix flatter

because there's only 1 attr left now

* devshell: fix pyEnv by filtering editables

from the environment.

* devshell: skip existing editables

* devshell: remove patched pyEnv, rewrite sys.path

* devshell: add special case for root package

* python-local-development: improve editable example

* python-local-development: filter source

* devshell: reset site_dir on each load

* pip: don't ignoreCollisions in pyEnv

* pip: default to no sitecustomize.py...

and teach the devshell how to load it

* pip: rewrite editable in python

* editable: use shutil.copytree

* editable: refactor into functions

* editables: add suport for console_scripts

* improve pip editables interface

- change editables interface to use bool or path
- expose public.shellHook for composition
- use shellHook by default in public.devShell
- extend example to include a script

* improve edtiable support:

- remove dependency on root package build
- always set root package as editable by default
- compute dit-info dir by calling a packages build backend or extracting directly from the wheel
- only pass required sources to editables shell hook
- add build inputs of all editables to teh dev shell
- move editables state to .dream2nix/python to remove likelyhood of collision with other ecosystems state
- delete editables state if editables configuration changed

* refine python editable support:

- never copy dependency sources
- force user to specify a local path

---------

Co-authored-by: DavHau <hsngrmpf+github@gmail.com>
2024-05-17 01:00:09 +02:00
Jairo Llopis
4cd8cbdb20 feat(buildPythonPackage): support new pyproject format
https://github.com/NixOS/nixpkgs/pull/271597 implemented a new format for python packages that is starting to get traction in nixpkgs/unstable (soon to become NixOS 24.05).

That is a progress towards https://github.com/NixOS/nixpkgs/issues/272178, which would be a massive improvement for dream2nix once completed.

This first step just makes sure that `buildPythonPackage` supports the new arguments properly. Without this change, many dream2nix python derivations fail to build on nixos-unstable right now.

Probably other python auto-builders should leverage these new options when available for the d2n user. That can be added later.

@moduon MT-1075
2024-05-13 09:40:15 +01:00
Peter Ferenczy
32aa32d397 fmt 2024-04-28 21:46:53 +07:00
Peter Ferenczy
62c6361657 mkDefault 2024-04-28 21:46:53 +07:00
Peter Ferenczy
3e4cd42a9f add rsync to nodejs-devshell-v3 2024-04-28 21:46:53 +07:00
purepani
babb6e3b69 python/pdm: Changes deps.python3 refs to deps.python 2024-04-28 13:18:00 +07:00
phaer
c5cfd416ab core.lock: Add missing coreutils dependency...
otherwise we implictly assume realpath to already
be in $PATH which isn't always the case, i.e. on older
macOS releases.
2024-04-15 19:30:05 +07:00
phaer
d3f8dbfc79 fix typo 2024-04-15 19:30:05 +07:00
shivaraj-bh
49f14262e3 WIP-python-pdm: add public.pyEnv; reuse public.pyEnv in public.devShell 2024-04-08 18:24:01 +07:00
DavHau
2fc4ef33cc docs: overhaul reference documentation pipeline
- use nixosOptionsDoc commonMark output
- simplify a lot of things
- redesign towards a future generic standalon nixosOptionsWebsite function
2024-04-05 19:42:14 +07:00
lbodor
26a6c3a340 WIP-haskell-cabal: fetch correct cabal revisions
Fetch cabal revisions from `https://hackage.haskell.org/package/{name}-{version}/revisions/`, rather
than `https://hackage.haskell.org/package/{name}/revisions/`, which fetches revisions for the latest
package version, not the version pinned in `./dist-newstyle/cache/plan.json`.
2024-03-29 03:56:14 +00:00
purepani
4710341e67 python/pdm: Fixes order of build inputs in devShell so that the path to
path to the interpreter is correct.
2024-03-29 02:50:26 +00:00
DavHau
6ddb9a7f5e override/python: fix devShells for nvidia packages 2024-03-26 05:53:26 +00:00
DavHau
306e5bff07 pythonEnv: init
Add experimental one-liner command to create python environments.

```command
$ nix run github:nix-community/dream2nix#__pythonEnv requests whisper-ctranslate2
...

$ python -c "import requests; print('imported requests')"
imported requests

$ whisper-ctranslate2 --version
whisper-ctranslate2 0.4.1
```
2024-03-26 05:03:45 +00:00
Robert Hensing
d623e6ec21 mkDerivation: fix typo in checkFlags 2024-03-22 21:02:23 +07:00
DavHau
f017de2de4 mkDerivation: stop depending on all core modules
mkDerivation should be minimal, but currently it always pulls in modules like `lock` and `paths` etc. despite those not being needed for many instances, like when mkDerivation is imported by a dependency.

Modules like pip which do need some of these core modules, now need to directly depend on them.

This will remove a bunch of unneeded options which currently pollute the manual.

The remaining core dependencies of mkDerivation are now: deps, env, ui
2024-03-18 11:15:30 +00:00
DavHau
f15cc37662 mkDerivation/builtins-derivation: refactor: implementation.nix -> default.nix 2024-03-17 22:55:10 +07:00
DavHau
6bbac51494 python/pdm: hide groups as implementation detail
The most important feature of the groups module was the overriding which is a separate module now. Therefore removing the fact that groups are used from the docs to decrease the API surface to maintain.

Instead render the override and overrideAll options in the manual of pdm correctly by setting `overrideType`.
2024-03-17 15:48:07 +00:00
DavHau
41a1973ed3 groups: remove options overrides and overrideAll
...since those are now managed by the dedicated overrides module
2024-03-17 14:37:33 +00:00
DavHau
86e4b94689 docs/overrides: explain how to find options 2024-03-17 14:34:12 +00:00
DavHau
b6e6d36f9d docs: init overrides 2024-03-17 09:27:24 +00:00
DavHau
b8d00b7c98 docs: update module system intro 2024-03-17 06:04:46 +00:00
DavHau
b727a07775 override: unify overriding across modules
Add options `overrides.${name}` and `overrideAll` to all language modules that manage a dependency tree.

This is done in a backward compatible fashion. Old options for overriding dependencies continue working, though those are not displayed in the manual anymore.

For the following modules the manual now suggest using the new unified override options:
- for pip: use `pip.overrides` and `pip.overrideAll` instead of  `pip.drvs`
- for nodejs-granular-v3: use `nodejs-granular-v3.overrides` and `nodejs-granular-v3.overrideAll` instead of `nodejs-granular-v3.deps`
- `php-granular`: use `php-granular.override` and `php-granular.overrideAll` instead of `php-granular.deps`
2024-03-13 14:25:41 +07:00
DavHau
63ab24d893 docs: refine public/experimental
Move some barely used modules to the experimental section to reduce surface.
Instead of the separate dev-shell module the nodejs package should have a devShell subattribute.
2024-03-12 11:25:12 +00:00
DavHau
45d941d49f docs: don't strip foreign module options
previously the options of other imported modules were not shown which gave an incomplete picture. For example the docs page for the `pip` module did not show the options prefixed with `mkDerivation` which are accepted and needed by the pip module.
It's better to have everything on one page for each module.
2024-03-12 11:05:59 +00:00
DavHau
ba4be52cdf ci: build website only on x86_64-linux to reduce build time
The website builds really slow on other platforms due to the buildbot remote builder setup.
2024-03-12 10:08:01 +00:00
DavHau
211528826c cleanup core options
- don't import eval-cache and flags modules by default (those have never been used so far)
- mark options of package-func as internal. Those are for maintainers only and don't need to be rendered by default
2024-03-12 09:41:59 +00:00
DavHau
bb13da673a checks.nix-unit: use nix-unit from nixpkgs 2024-03-12 07:09:04 +00:00
DavHau
63c9f51a43 pdm: add test for jupyterlab 2024-03-11 14:24:27 +00:00
DavHau
deb4c98014 docs: fix links to examples 2024-03-11 10:41:05 +00:00
DavHau
42838c5909 examples: cleanup and rename to simplify
- name example more consistently, eg. `{language}-packaging-{feature}` or `{language}-local-development-{feature}`
- move some examples to the modules integration tests directory instead as their purpose was mainly testing and they weren't good example
- module owned checks: import via flake if available
2024-03-09 04:45:59 +00:00
DavHau
3b9e0780a0 docs: separate public and internal modules 2024-03-07 08:51:00 +00:00
DavHau
5f6c966ab3 update-flake: fix to work with multiple directories once 2024-03-07 05:22:52 +00:00
DavHau
e8f018187c fix collision issue with deps.bash 2024-02-28 07:08:39 +00:00
DavHau
887022b8c1 update nixpkgs:
- downgrade mitmproxy to avoid breakage
- add test for pip lock script
- fix bug in flake compat implementation (https://github.com/nix-community/flake-compat/pull/4#issuecomment-1950301370)
- small fix in docs for lock and eval-cache
- fix lock refresh script: don't swallow errors
2024-02-24 13:47:45 +00:00
DavHau
3308e05808 pdm: add rudimentary devShell
Also fix lock script to not swallow error messages
2024-02-20 11:53:54 +00:00
DavHau
5a5ae25b04 fetching: set shallow=true for all fetchGit calls
This should lead to better performance
Also set allRefs=true in some places, as this reduces the risk of running into issues with older nix versions
2024-02-18 14:08:51 +07:00
Johannes Kirschbauer
3065f50344 nodejs: handle more npm dependency types 2024-02-16 09:44:32 +00:00
Johannes Kirschbauer
eed2342f7f nodejs: move installation into derivation module 2024-02-16 03:59:35 +00:00
Johannes Kirschbauer
10b523238c nodejs: move outputs in seperate module files 2024-02-15 06:42:10 +00:00
Arnout Engelen
367e7fcc98 Getting started docs
I always forget how to get going with dream2nix, and while this
page doesn't contain much more than you'd already find in the
examples, I do think it's a good starting point for allowing
people to start more confidently.
2024-02-09 20:33:18 +07:00
DavHau
16f2c654c6 pdm: stop using config.deps.python3 in favor of config.deps.python 2024-01-24 08:09:56 +00:00
DavHau
0a30902e27 pdm: fix various bugs
- default to correct version of setuptools from lock file
- better default for python3 set in dream2nix.core.paths
2024-01-24 07:29:32 +00:00
Jairo Llopis
233c4d8106 fix(nodejs-package-json-v3): remove duplicate dependency 2024-01-23 20:32:36 +07:00
Vincent Vanlaer
eb3b7028d5 pip: allow git+ssh dependencies 2024-01-23 20:31:31 +07:00
Herman Fries
774019d486 lock: Extend environment instead of replacing it
This allows running impure commands, that rely on user's configuration. E.g. to access a private gitlab project.
2024-01-20 02:05:05 +00:00