git.iko.soy/typeable/nixpkgs
on master help sign in
doc: Rewrite python onboarding tutorials in manual

Based on some feedback in #87094 and discussion with @fridh, this re-organizes the onboarding tutorial in the Nixpkgs manual's python section, so that we start with the simplest, most ad-hoc examples and work our way up. This progresses from: 1. How to create an temporary python env at the cmdline, then 2. How to create a specific python env for a single script, then 3. How to create a specific python env for a project in a shell.nix, then 4. How to install a specific python env globally on the system or in a user profile. Additionally, I've tried to standardize on some of the "best practice" ways of doing things: 1. Instead of saying that this command style is "supported but strongly not discouraged", I've just deleted it to avoid confusion. Bad: nix-shell -p python38Packages.numpy python38Packages.toolz Good: nix-shell -p 'python38.withPackages(ps: with ps; [ numpy toolz ])' 2. In the portion where we show how to add stuff to the user's `XDG_CONFIG_HOME`, use overlays instead of `config.nix`. The former can do everything the latter can do, but is also much more generic and powerful, because it can compose with other files, compose with other envs, compose with overlays that do things like swap whether tensorflow and pytorch are built openblas/mkl/cuda stacks, and so on. The user is eventually going to see the overlay, so to avoid confusion let's standardize on it.

commitc88290789c06e19453375ad85f3cd1d417b07ae9 authorBenjamin Hipple <bhipple@protonmail.com> dateSat 9 May 2020, 19:11 (7 years ago) parent8476545
1 file changed+254−138
doc/languages-frameworks/python.section.md+254−138
doc/languages-frameworks/python.section.md+254−138
@@ -36,121 +36,191 @@ The Nix and NixOS manuals explain how packages are generally installed. In the
3636case of Python and Nix, it is important to make a distinction between whether the
3737package is considered an application or a library.
3838
39−Applications on Nix are typically installed into your user
40−profile imperatively using `nix-env -i`, and on NixOS declaratively by adding the
41−package name to `environment.systemPackages` in `/etc/nixos/configuration.nix`.
42−Dependencies such as libraries are automatically installed and should not be
43−installed explicitly.
44−
45−The same goes for Python applications and libraries. Python applications can be
46−installed in your profile. But Python libraries you would like to use for
47−development cannot be installed, at least not individually, because they won't
48−be able to find each other resulting in import errors. Instead, it is possible
49−to create an environment with `python.buildEnv` or `python.withPackages` where
50−the interpreter and other executables are able to find each other and all of the
51−modules.
52−
53−In the following examples we create an environment with Python 3.8, `numpy` and
54−`toolz`. As you may imagine, there is one limitation here, and that's that
55−you can install only one environment at a time. You will notice the complaints
56−about collisions when you try to install a second environment.
57−
58−##### Environment defined in separate `.nix` file
59−
60−Create a file, e.g. `build.nix`, with the following expression
61−```nix
62−with import <nixpkgs> {};
39+Applications on Nix are typically installed into your user profile imperatively
40+using `nix-env -i`, and on NixOS declaratively by adding the package name to
41+`environment.systemPackages` in `/etc/nixos/configuration.nix`. Dependencies
42+such as libraries are automatically installed and should not be installed
43+explicitly.
6344
64−python38.withPackages (ps: with ps; [ numpy toolz ])
65−```
66−and install it in your profile with
67−```shell
68−nix-env -if build.nix
69−```
70−Now you can use the Python interpreter, as well as the extra packages (`numpy`,
71−`toolz`) that you added to the environment.
45+The same goes for Python applications. Python applications can be installed in
46+your profile, and will be wrapped to find their exact library dependencies,
47+without impacting other applications or polluting your user environment.
7248
73−##### Environment defined in `~/.config/nixpkgs/config.nix`
49+But Python libraries you would like to use for development cannot be installed,
50+at least not individually, because they won't be able to find each other
51+resulting in import errors. Instead, it is possible to create an environment
52+with `python.buildEnv` or `python.withPackages` where the interpreter and other
53+executables are wrapped to be able to find each other and all of the modules.
7454
75−If you prefer you could also add the environment as a package override to the
76−Nixpkgs set, e.g. using `config.nix`,
55+In the following examples we will start by creating a simple, ad-hoc environment
56+with a nix-shell that has `numpy` and `toolz` in Python 3.8; then we will create
57+a re-usable environment in a single-file Python script; then we will create a
58+full Python environment for development with this same environment.
7759
78−```nix
79−{ # ...
60+Philosphically, this should be familiar to users who are used to a `venv` style
61+of development: individual projects create their own Python environments without
62+impacting the global environment or each other.
8063
81− packageOverrides = pkgs: with pkgs; {
82− myEnv = python38.withPackages (ps: with ps; [ numpy toolz ]);
83− };
84−}
85−```
86−and install it in your profile with
87−
88−```shell
89−nix-env -iA nixpkgs.myEnv
90−```
64+#### Ad-hoc temporary Python environment with `nix-shell`
9165
92−The environment is is installed by referring to the attribute, and considering
93−the `nixpkgs` channel was used.
66+The simplest way to start playing with the way nix wraps and sets up Python
67+environments is with `nix-shell` at the cmdline. These environments create a
68+temporary shell session with a Python and a *precise* list of packages (plus
69+their runtime dependencies), with no other Python packages in the Python
70+interpreter's scope.
9471
95−##### Environment defined in `/etc/nixos/configuration.nix`
72+To create a Python 3.8 session with `numpy` and `toolz` available, run:
9673
97−For the sake of completeness, here's another example how to install the
98−environment system-wide.
74+```sh
75+$ nix-shell -p 'python38.withPackages(ps: with ps; [ numpy toolz ])'
76+```
9977
100−```nix
101−{ # ...
78+By default `nix-shell` will start a `bash` session with this interpreter in our
79+`PATH`, so if we then run:
10280
103− environment.systemPackages = with pkgs; [
104− (python38.withPackages(ps: with ps; [ numpy toolz ]))
105− ];
106−}
81+```
82+[nix-shell:~/src/nixpkgs]$ python3
83+Python 3.8.1 (default, Dec 18 2019, 19:06:26)
84+[GCC 9.2.0] on linux
85+Type "help", "copyright", "credits" or "license" for more information.
86+>>> import numpy; import toolz
10787```
10888
109−#### Temporary Python environment with `nix-shell`
89+Note that no other modules are in scope, even if they were imperatively
90+installed into our user environment as a dependency of a Python application:
11091
111−The examples in the previous section showed how to install a Python environment
112−into a profile. For development you may need to use multiple environments.
113−`nix-shell` gives the possibility to temporarily load another environment, akin
114−to `virtualenv`.
92+```
93+>>> import requests
94+Traceback (most recent call last):
95+ File "<stdin>", line 1, in <module>
96+ModuleNotFoundError: No module named 'requests'
97+```
11598
116−There are two methods for loading a shell with Python packages. The first and
117−recommended method is to create an environment with `python.buildEnv` or
118−`python.withPackages` and load that. E.g.
99+We can add as many additional modules onto the `nix-shell` as we need, and we
100+will still get 1 wrapped Python interpreter. We can start the interpreter
101+directly like so:
119102
120103```sh
121−$ nix-shell -p 'python38.withPackages(ps: with ps; [ numpy toolz ])'
104+$ nix-shell -p 'python38.withPackages(ps: with ps; [ numpy toolz requests ])' --run python3
105+these derivations will be built:
106+ /nix/store/xbdsrqrsfa1yva5s7pzsra8k08gxlbz1-python3-3.8.1-env.drv
107+building '/nix/store/xbdsrqrsfa1yva5s7pzsra8k08gxlbz1-python3-3.8.1-env.drv'...
108+created 277 symlinks in user environment
109+Python 3.8.1 (default, Dec 18 2019, 19:06:26)
110+[GCC 9.2.0] on linux
111+Type "help", "copyright", "credits" or "license" for more information.
112+>>> import requests
113+>>>
114+```
115+
116+Notice that this time it built a new Python environment, which now includes
117+`requests`. Building an environment just creates wrapper scripts that expose the
118+selected dependencies to the interpreter while re-using the actual modules. This
119+means if any other env has installed `requests` or `numpy` in a different
120+context, we don't need to recompile them -- we just recompile the wrapper script
121+that sets up an interpreter pointing to them. This matters much more for "big"
122+modules like `pytorch` or `tensorflow`.
123+
124+Module names usually match their names on [pypi.org](https://pypi.org/), but
125+you can use the [Nixpkgs search website](https://nixos.org/nixos/packages.html)
126+to find them as well (along with non-python packages).
127+
128+At this point we can create throwaway experimental Python environments with
129+arbitrary dependencies. This is a good way to get a feel for how the Python
130+interpreter and dependencies work in Nix and NixOS, but to do some actual
131+development, we'll want to make it a bit more persistent.
132+
133+##### Running Python scripts and using `nix-shell` as shebang
134+
135+Sometimes, we have a script whose header looks like this:
136+
137+```python
138+#!/usr/bin/env python3
139+import numpy as np
140+a = np.array([1,2])
141+b = np.array([3,4])
142+print(f"The dot product of {a} and {b} is: {np.dot(a, b)}")
122143```
123144
124−opens a shell from which you can launch the interpreter
145+Executing this script requires a `python3` that has `numpy`. Using what we learned
146+in the previous section, we could startup a shell and just run it like so:
125147
126−```sh
127−[nix-shell:~] python3
148+```
149+nix-shell -p 'python38.withPackages(ps: with ps; [ numpy ])' --run 'python3 foo.py'
150+The dot product of [1 2] and [3 4] is: 11
151+```
152+
153+But if we maintain the script ourselves, and if there are more dependencies, it
154+may be nice to encode those dependencies in source to make the script re-usable
155+without that bit of knowledge. That can be done by using `nix-shell` as a
156+[shebang](https://en.wikipedia.org/wiki/Shebang_(Unix), like so:
157+
158+```python
159+#!/usr/bin/env nix-shell
160+#!nix-shell -i python3 -p "python3.withPackages(ps: [ ps.numpy ])"
161+import numpy as np
162+a = np.array([1,2])
163+b = np.array([3,4])
164+print(f"The dot product of {a} and {b} is: {np.dot(a, b)}")
128165```
129166
130−The other method, which is not recommended, does not create an environment and
131−requires you to list the packages directly,
167+Then we simply execute it, without requiring any environment setup at all!
132168
133169```sh
134−$ nix-shell -p python38.pkgs.numpy python38.pkgs.toolz
170+$ ./foo.py
171+The dot product of [1 2] and [3 4] is: 11
135172```
136173
137−Again, it is possible to launch the interpreter from the shell. The Python
138−interpreter has the attribute `pkgs` which contains all Python libraries for
139−that specific interpreter.
174+If the dependencies are not available on the host where `foo.py` is executed, it
175+will build or download them from a Nix binary cache prior to starting up, prior
176+that it is executed on a machine with a multi-user nix installation.
177+
178+This provides a way to ship a self bootstrapping Python script, akin to a
179+statically linked binary, where it can be run on any machine (provided nix is
180+installed) without having to assume that `numpy` is installed globally on the
181+system.
182+
183+By default it is pulling the import checkout of Nixpkgs itself from our nix
184+channel, which is nice as it cache aligns with our other package builds, but we
185+can make it fully reproducible by pinning the `nixpkgs` import:
186+
187+```python
188+#!/usr/bin/env nix-shell
189+#!nix-shell -i python3 -p "python3.withPackages(ps: [ ps.numpy ])"
190+#!nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/d373d80b1207d52621961b16aa4a3438e4f98167.tar.gz
191+import numpy as np
192+a = np.array([1,2])
193+b = np.array([3,4])
194+print(f"The dot product of {a} and {b} is: {np.dot(a, b)}")
195+```
196+
197+This will execute with the exact same versions of Python 3.8, numpy, and system
198+dependencies a year from now as it does today, because it will always use
199+exactly git commit `d373d80b1207d52621961b16aa4a3438e4f98167` of Nixpkgs for all
200+of the package versions.
201+
202+This is also a great way to ensure the script executes identically on different
203+servers.
140204
141205##### Load environment from `.nix` expression
142−As explained in the Nix manual, `nix-shell` can also load an
143−expression from a `.nix` file. Say we want to have Python 3.8, `numpy`
144−and `toolz`, like before, in an environment. Consider a `shell.nix` file
145−with
206+
207+We've now seen how to create an ad-hoc temporary shell session, and how to
208+create a single script with Python dependencies, but in the course of normal
209+development we're usually working in an entire package repository.
210+
211+As explained in the Nix manual, `nix-shell` can also load an expression from a
212+`.nix` file. Say we want to have Python 3.8, `numpy` and `toolz`, like before,
213+in an environment. We can add a `shell.nix` file describing our dependencies:
146214
147215```nix
148216with import <nixpkgs> {};
149−
150217(python38.withPackages (ps: [ps.numpy ps.toolz])).env
151218```
152219
153−Executing `nix-shell` gives you again a Nix shell from which you can run Python.
220+And then at the command line, just typing `nix-shell` produces the same
221+environment as before. In a normal project, we'll likely have many more
222+dependencies; this can provide a way for developers to share the environments
223+with each other and with CI builders.
154224
155225What's happening here?
156226
@@ -158,9 +228,9 @@ What's happening here?
158228 imports the `<nixpkgs>` function, `{}` calls it and the `with` statement
159229 brings all attributes of `nixpkgs` in the local scope. These attributes form
160230 the main package set.
161−2. Then we create a Python 3.8 environment with the `withPackages` function.
231+2. Then we create a Python 3.8 environment with the `withPackages` function, as before.
1622323. The `withPackages` function expects us to provide a function as an argument
163− that takes the set of all python packages and returns a list of packages to
233+ that takes the set of all Python packages and returns a list of packages to
164234 include in the environment. Here, we select the packages `numpy` and `toolz`
165235 from the package set.
166236
@@ -168,7 +238,6 @@ To combine this with `mkShell` you can:
168238
169239```nix
170240with import <nixpkgs> {};
171−
172241let
173242 pythonEnv = python38.withPackages (ps: [
174243 ps.numpy
@@ -177,50 +246,98 @@ let
177246in mkShell {
178247 buildInputs = [
179248 pythonEnv
180− hello
249+
250+ black
251+ mypy
252+
253+ libffi
254+ openssl
181255 ];
182256}
183257```
184258
185−##### Execute command with `--run`
186−A convenient option with `nix-shell` is the `--run`
187−option, with which you can execute a command in the `nix-shell`. We can
188−e.g. directly open a Python shell
259+This will create a unified environment that has not just our Python interpreter
260+and its Python dependencies, but also tools like `black` or `mypy` and libraries
261+like `libffi` the `openssl` in scope. This is generic and can span any number of
262+tools or languages across the Nixpkgs ecosystem.
189263
190−```sh
191−$ nix-shell -p python38Packages.numpy python38Packages.toolz --run "python3"
264+##### Installing environments globally on the system
265+
266+Up to now, we've been creating environments scoped to an ad-hoc shell session,
267+or a single script, or a single project. This is generally advisable, as it
268+avoids pollution across contexts.
269+
270+However, sometimes we know we will often want a Python with some basic packages,
271+and want this available without having to enter into a shell or build context.
272+This can be useful to have things like vim/emacs editors and plugins or shell
273+tools "just work" without having to set them up, or when running other software
274+that expects packages to be installed globally.
275+
276+To create your own custom environment, create a file in `~/.config/nixpkgs/overlays/`
277+that looks like this:
278+
279+```nix
280+# ~/.config/nixpkgs/overlays/myEnv.nix
281+self: super: {
282+ myEnv = super.buildEnv {
283+ name = "myEnv";
284+ paths = [
285+ # A Python 3 interpreter with some packages
286+ (self.python3.withPackages (
287+ ps: with ps; [
288+ pyflakes
289+ pytest
290+ python-language-server
291+ ]
292+ ))
293+
294+ # Some other packages we'd like as part of this env
295+ self.mypy
296+ self.black
297+ self.ripgrep
298+ self.tmux
299+ ];
300+ };
301+}
192302```
193303
194−or run a script
304+You can then build and install this to your profile with:
195305
196306```sh
197−$ nix-shell -p python38Packages.numpy python38Packages.toolz --run "python3 myscript.py"
307+nix-env -iA myEnv
198308```
199309
200−##### `nix-shell` as shebang
201−In fact, for the second use case, there is a more convenient method. You can add
202−a [shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) to your script
203−specifying which dependencies `nix-shell` needs. With the following shebang, you
204−can just execute `./myscript.py`, and it will make available all dependencies
205−and run the script in the `python3` shell.
310+One limitation of this is that you can only have 1 Python env installed
311+globally, since they conflict on the `python` to load out of your `PATH`.
312+
313+If you get a conflict or prefer to keep the setup clean, you can have `nix-env`
314+atomically *uninstall* all other imperatively installed packages and replace
315+your profile with just `myEnv` by using the `--replace` flag.
316+
317+##### Environment defined in `/etc/nixos/configuration.nix`
206318
207−```py
208−#! /usr/bin/env nix-shell
209−#! nix-shell -i python3 -p "python3.withPackages(ps: [ps.numpy])"
319+For the sake of completeness, here's how to install the environment system-wide
320+on NixOS.
210321
211−import numpy
322+```nix
323+{ # ...
212324
213−print(numpy.__version__)
325+ environment.systemPackages = with pkgs; [
326+ (python38.withPackages(ps: with ps; [ numpy toolz ]))
327+ ];
328+}
214329```
215330
216331### Developing with Python
217332
218−Now that you know how to get a working Python environment with Nix, it is time
219−to go forward and start actually developing with Python. We will first have a
220−look at how Python packages are packaged on Nix. Then, we will look at how you
221−can use development mode with your code.
333+Above, we were mostly just focused on use cases and what to do to get started
334+creating working Python environments in nix.
222335
223−#### Packaging a library
336+Now that you know the basics to be up and running, it is time to take a step
337+back and take a deeper look at at how Python packages are packaged on Nix. Then,
338+we will look at how you can use development mode with your code.
339+
340+#### Python library packages in Nixpkgs
224341
225342With Nix all packages are built by functions. The main function in Nix for
226343building Python libraries is `buildPythonPackage`. Let's see how we can build the
@@ -262,6 +379,7 @@ An expression for `toolz` can be found in the Nixpkgs repository. As explained
262379in the introduction of this Python section, a derivation of `toolz` is available
263380for each interpreter version, e.g. `python38.pkgs.toolz` refers to the `toolz`
264381derivation corresponding to the CPython 3.8 interpreter.
382+
265383The above example works when you're directly working on
266384`pkgs/top-level/python-packages.nix` in the Nixpkgs repository. Often though,
267385you will want to test a Nix expression outside of the Nixpkgs tree.
@@ -312,7 +430,7 @@ Our example, `toolz`, does not have any dependencies on other Python packages or
312430system libraries. According to the manual, `buildPythonPackage` uses the
313431arguments `buildInputs` and `propagatedBuildInputs` to specify dependencies. If
314432something is exclusively a build-time dependency, then the dependency should be
315−included as a `buildInput`, but if it is (also) a runtime dependency, then it
433+included in `buildInputs`, but if it is (also) a runtime dependency, then it
316434should be added to `propagatedBuildInputs`. Test dependencies are considered
317435build-time dependencies and passed to `checkInputs`.
318436
@@ -423,10 +541,11 @@ Note also the line `doCheck = false;`, we explicitly disabled running the test-s
423541
424542#### Develop local package
425543
426−As a Python developer you're likely aware of [development mode](http://setuptools.readthedocs.io/en/latest/setuptools.html#development-mode) (`python setup.py develop`);
427−instead of installing the package this command creates a special link to the project code.
428−That way, you can run updated code without having to reinstall after each and every change you make.
429−Development mode is also available. Let's see how you can use it.
544+As a Python developer you're likely aware of [development mode](http://setuptools.readthedocs.io/en/latest/setuptools.html#development-mode)
545+(`python setup.py develop`); instead of installing the package this command
546+creates a special link to the project code. That way, you can run updated code
547+without having to reinstall after each and every change you make. Development
548+mode is also available. Let's see how you can use it.
430549
431550In the previous Nix expression the source was fetched from an url. We can also
432551refer to a local source instead using `src = ./path/to/source/tree;`
@@ -455,7 +574,6 @@ buildPythonPackage rec {
455574It is important to note that due to how development mode is implemented on Nix
456575it is not possible to have multiple packages simultaneously in development mode.
457576
458−
459577### Organising your packages
460578
461579So far we discussed how you can use Python on Nix, and how you can develop with
@@ -481,11 +599,11 @@ We first create a function that builds `toolz` in `~/path/to/toolz/release.nix`
481599
482600buildPythonPackage rec {
483601 pname = "toolz";
484− version = "0.7.4";
602+ version = "0.10.0";
485603
486604 src = fetchPypi {
487605 inherit pname version;
488− sha256 = "43c2c9e5e7a16b6c88ba3088a9bfc82f7db8e13378be7c78d6c14a5f8ed05afd";
606+ sha256 = "08fdd5ef7c96480ad11c12d472de21acd32359996f69a5259299b540feba4560";
489607 };
490608
491609 meta = with lib; {
@@ -497,8 +615,8 @@ buildPythonPackage rec {
497615}
498616```
499617
500−It takes an argument `buildPythonPackage`.
501−We now call this function using `callPackage` in the definition of our environment
618+It takes an argument `buildPythonPackage`. We now call this function using
619+`callPackage` in the definition of our environment
502620
503621```nix
504622with import <nixpkgs> {};
@@ -548,7 +666,7 @@ Each interpreter has the following attributes:
548666- `buildEnv`. Function to build python interpreter environments with extra packages bundled together. See section *python.buildEnv function* for usage and documentation.
549667- `withPackages`. Simpler interface to `buildEnv`. See section *python.withPackages function* for usage and documentation.
550668- `sitePackages`. Alias for `lib/${libPrefix}/site-packages`.
551−- `executable`. Name of the interpreter executable, e.g. `python3.7`.
669+- `executable`. Name of the interpreter executable, e.g. `python3.8`.
552670- `pkgs`. Set of Python packages for that specific interpreter. The package set can be modified by overriding the interpreter and passing `packageOverrides`.
553671
554672### Building packages and applications
@@ -643,7 +761,7 @@ following are specific to `buildPythonPackage`:
643761 appears more than once in dependency tree. Default is `true`.
644762* `disabled` ? false: If `true`, package is not built for the particular Python
645763 interpreter version.
646−* `dontWrapPythonPrograms ? false`: Skip wrapping of python programs.
764+* `dontWrapPythonPrograms ? false`: Skip wrapping of Python programs.
647765* `permitUserSite ? false`: Skip setting the `PYTHONNOUSERSITE` environment
648766 variable in wrapped programs.
649767* `installFlags ? []`: A list of strings. Arguments to be passed to `pip
@@ -730,7 +848,7 @@ Another difference is that `buildPythonPackage` by default prefixes the names of
730848the packages with the version of the interpreter. Because this is irrelevant for
731849applications, the prefix is omitted.
732850
733−When packaging a python application with `buildPythonApplication`, it should be
851+When packaging a Python application with `buildPythonApplication`, it should be
734852called with `callPackage` and passed `python` or `pythonPackages` (possibly
735853specifying an interpreter version), like this:
736854
@@ -761,7 +879,7 @@ luigi = callPackage ../applications/networking/cluster/luigi { };
761879```
762880
763881Since the package is an application, a consumer doesn't need to care about
764−python versions or modules, which is why they don't go in `pythonPackages`.
882+Python versions or modules, which is why they don't go in `pythonPackages`.
765883
766884#### `toPythonApplication` function
767885
@@ -875,7 +993,7 @@ thus be also written like this:
875993```nix
876994with import <nixpkgs> {};
877995
878−(python36.withPackages (ps: [ps.numpy ps.requests])).env
996+(python38.withPackages (ps: [ps.numpy ps.requests])).env
879997```
880998
881999In contrast to `python.buildEnv`, `python.withPackages` does not support the
@@ -932,7 +1050,7 @@ pythonPackages.buildPythonPackage {
9321050Running `nix-shell` with no arguments should give you the environment in which
9331051the package would be built with `nix-build`.
9341052
935−Shortcut to setup environments with C headers/libraries and python packages:
1053+Shortcut to setup environments with C headers/libraries and Python packages:
9361054
9371055```shell
9381056nix-shell -p pythonPackages.pyramid zlib libjpeg git
@@ -963,7 +1081,6 @@ have timestamp 1. The `buildPythonPackage` function sets `DETERMINISTIC_BUILD=1`
9631081and [PYTHONHASHSEED=0](https://docs.python.org/3.8/using/cmdline.html#envvar-PYTHONHASHSEED).
9641082Both are also exported in `nix-shell`.
9651083
966−
9671084### Automatic tests
9681085
9691086It is recommended to test packages as part of the build process.
@@ -976,7 +1093,7 @@ example of such a situation is when `py.test` is used.
9761093#### Common issues
9771094
9781095* Non-working tests can often be deselected. By default `buildPythonPackage`
979− runs `python setup.py test`. Most python modules follows the standard test
1096+ runs `python setup.py test`. Most Python modules follows the standard test
9801097 protocol where the pytest runner can be used instead. `py.test` supports a
9811098 `-k` parameter to ignore test methods or classes:
9821099
@@ -1052,7 +1169,7 @@ let
10521169 newpkgs = import pkgs.path { overlays = [ (self: super: {
10531170 python38 = let
10541171 packageOverrides = python-self: python-super: {
1055− numpy = python-super.numpy_1_18.3;
1172+ numpy = python-super.numpy_1_18;
10561173 };
10571174 in super.python38.override {inherit packageOverrides;};
10581175 } ) ]; };
@@ -1127,14 +1244,14 @@ If you want to create a Python environment for development, then the recommended
11271244method is to use `nix-shell`, either with or without the `python.buildEnv`
11281245function.
11291246
1130−### How to consume python modules using pip in a virtual environment like I am used to on other Operating Systems?
1247+### How to consume Python modules using pip in a virtual environment like I am used to on other Operating Systems?
11311248
11321249While this approach is not very idiomatic from Nix perspective, it can still be
11331250useful when dealing with pre-existing projects or in situations where it's not
11341251feasible or desired to write derivations for all required dependencies.
11351252
11361253This is an example of a `default.nix` for a `nix-shell`, which allows to consume
1137−a virtual environment created by `venv`, and install python modules through
1254+a virtual environment created by `venv`, and install Python modules through
11381255`pip` the traditional way.
11391256
11401257Create this `default.nix` file, together with a `requirements.txt` and simply
@@ -1149,7 +1266,7 @@ in pkgs.mkShell rec {
11491266 name = "impurePythonEnv";
11501267 venvDir = "./.venv";
11511268 buildInputs = [
1152− # A python interpreter including the 'venv' module is required to bootstrap
1269+ # A Python interpreter including the 'venv' module is required to bootstrap
11531270 # the environment.
11541271 pythonPackages.python
11551272
@@ -1163,7 +1280,7 @@ in pkgs.mkShell rec {
11631280 pythonPackages.requests
11641281
11651282 # In this particular example, in order to compile any binary extensions they may
1166− # require, the python modules listed in the hypothetical requirements.txt need
1283+ # require, the Python modules listed in the hypothetical requirements.txt need
11671284 # the following packages to be installed locally:
11681285 taglib
11691286 openssl
@@ -1183,7 +1300,7 @@ in pkgs.mkShell rec {
11831300}
11841301```
11851302
1186−In case the supplied venvShellHook is insufficient, or when python 2 support is
1303+In case the supplied venvShellHook is insufficient, or when Python 2 support is
11871304needed, you can define your own shell hook and adapt to your needs like in the
11881305following example:
11891306
@@ -1229,7 +1346,7 @@ in pkgs.mkShell rec {
12291346```
12301347
12311348Note that the `pip install` is an imperative action. So every time `nix-shell`
1232−is executed it will attempt to download the python modules listed in
1349+is executed it will attempt to download the Python modules listed in
12331350requirements.txt. However these will be cached locally within the `virtualenv`
12341351folder and not downloaded again.
12351352
@@ -1290,9 +1407,8 @@ self: super: {
12901407
12911408### How to use Intel's MKL with numpy and scipy?
12921409
1293−MKL can be configured using an overlay. See the section “[Using
1294−overlays to configure
1295−alternatives](#sec-overlays-alternatives-blas-lapack)”.
1410+MKL can be configured using an overlay. See the section "[Using overlays to
1411+configure alternatives](#sec-overlays-alternatives-blas-lapack)".
12961412
12971413### What inputs do `setup_requires`, `install_requires` and `tests_require` map to?
12981414