Commit Graph

127 Commits

Author SHA1 Message Date
adisbladis
9c7ff7277c
importNpmLock.buildNodeModules: init
`importNpmLock.buildNodeModules` returns a derivation with a pre-built `node_modules` directory, as imported by `importNpmLock`.
This is to be used together with `importNpmLock.hooks.linkNodeModulesHook` to facilitate `nix-shell`/`nix develop` based development workflows:

```nix
pkgs.mkShell {
  packages = [
    importNpmLock.hooks.linkNodeModulesHook
    nodejs
  ];

  npmDeps = importNpmLock.buildNodeModules {
    npmRoot = ./.;
    inherit nodejs;
  };
}
```
will create a development shell where a `node_modules` directory is created & packages symlinked to the Nix store when activated.

This code is adapted from https://github.com/adisbladis/buildNodeModules
2024-08-29 06:12:07 -07:00
seth
278e1bfb89
yarnConfigHook: apply as a postConfigure hook
This follows in this example of npmConfigHook and allows for using this
hook alongside others that may override configurePhase
2024-08-28 23:30:55 -04:00
Johannes Kirschbauer
3ed5055f0f
importNpmLock: init fetcherOps per package 2024-08-13 09:38:37 +02:00
Sandro Jäckel
f4e18f55ef fetch-npm-deps: be less noisy
Before every package in node_modules was printed. Lets be a bit more
quiet and drop that.
2024-08-09 15:08:33 -04:00
github-actions[bot]
f20ed12f93
Merge master into staging-next 2024-07-25 12:01:18 +00:00
Adam Joseph
9cac7a7475 fetch-yarn-deps: improve diagnostic messages
When the hash of an url being fetched does not match the expected value, this
commit will cause fetch-yarn-deps to include the url in the error message to
assist debugging.
2024-07-25 11:37:44 +02:00
Adam Joseph
84a75e9488 fetchYarnDeps: fix broken fetching logic for github releases
When a dependency references a github *release* URL, that dependency must be
fetched using https rather than git, since github does not require that
release tarballs have any relationship whatsoever to the git history.

This commit causes them to be fetched using https, not git.

A test case (which fails prior to this commit, and passes afterwards) is included.
2024-07-25 11:37:39 +02:00
Randy Eckenrode
f9b7f4ec09
tree-wide: use top-level cctools 2024-07-17 22:36:19 -04:00
Doron Behar
2400268a38 fetch-yarn-deps: handle yarn.lock deps without a resolved url 2024-07-10 09:39:10 +03:00
Doron Behar
ee7cfec5aa doc/javascript: document yarn{Config,Build}Hook 2024-07-10 09:39:03 +03:00
Doron Behar
296556a320 yarnBuildHook: init 2024-07-10 09:39:02 +03:00
Doron Behar
a84f3ca3d8 yarnConfigHook: init 2024-07-10 09:39:02 +03:00
Doron Behar
1151b39c24 fetch-yarn-deps: format with nixfmt (RFC166) 2024-07-10 09:39:01 +03:00
Lily Foster
c678743f80 treewide: remove lilyinstarlight 2024-07-02 03:17:01 +02:00
Atemu
882cf18005 fetch-yarn-deps: reference files to be copied directly
This would cause any drv to be rebuilt when any part of the default.nix changes.
What we actually care about is the two JS files though, so simply reference them
directly.

Co-authored-by: Infinidoge <infinidoge@inx.moe>
2024-06-14 17:53:18 +02:00
Infinidoge
efff2acc5b
fetchYarnDeps: properly accept src argument 2024-06-14 11:18:10 -04:00
Lily Foster
ddb94deafa prefetch-npm-deps: switch to data-encoding 2024-04-22 23:14:15 +10:00
Lily Foster
c588edaf25 prefetch-npm-deps: remove runtime dependency on nix 2024-04-22 23:14:15 +10:00
Sandro Jäckel
6181939cdf
fixup-yarn-lock: split out from prefetch-yarn-deps, cleanup installing
To reduce dependencies (mainly nix-prefetch-git and through that git,
git-lfs) when we just need to fixup a lock file, eg when building electron.
This also tries to avoid needless rebuilds when eg. golang is updated.

Also this cleans up and combined the build/installPhase of both tools to
be a lot simpler.
2024-04-17 11:35:22 -04:00
stuebinm
ff1a94e523 treewide: add meta.mainProgram to packages with a single binary
The nixpkgs-unstable channel's programs.sqlite was used to identify
packages producing exactly one binary, and these automatically added
to their package definitions wherever possible.
2024-03-19 03:14:51 +01:00
Lily Foster
ae86a507ed npmHooks.npmInstallHook: ignore bundle deps when calculating files to install
This assumes that downstream users of `buildNpmPackage` would rather our
own built `node_modules` be copied to the output rather than only the
`bundleDependencies` specified in the `package.json` file.

Having the latter behavior seems unexpected and unintuitive, and would
not work as installing from an `npm pack` is intended to (since doing
that would not do a `rebuild` step on those dependencies and it would
skip reifying a full dependency tree).
2024-03-14 14:04:51 +01:00
adisbladis
b6e4b86809 importNpmLock: init
This is an alternative to `fetchNpmDeps` that is notably different in that it uses metadata from `package.json` & `package-lock.json` instead of specifying a fixed-output hash.

Notable features:
- IFD free.
- Only fetches a node dependency once. No massive FODs.
- Support for URL, Git and path dependencies.
- Uses most of the existing `npmHooks`

`importNpmLock` can be used _only_ in the cases where we need to check in a `package-lock.json` in the tree.
Currently this means that we have 13 packages that would be candidates to use this function, though I expect most usage to be in private repositories.

This is upstreaming the builder portion of https://github.com/adisbladis/buildNodeModules into nixpkgs (different naming but the code is the same).
I will archive this repository and consider nixpkgs the new upstream once it's been merged.

For more explanations and rationale see https://discourse.nixos.org/t/buildnodemodules-the-dumbest-node-to-nix-packaging-tool-yet/35733

Example usage:
``` nix
stdenv.mkDerivation {
  pname = "my-nodejs-app";
  version = "0.1.0";

  src = ./.;

  nativeBuildInputs = [
    importNpmLock.hooks.npmConfigHook
    nodejs
    nodejs.passthru.python # for node-gyp
    npmHooks.npmBuildHook
    npmHooks.npmInstallHook
  ];

  npmDeps = buildNodeModules.fetchNodeModules {
    npmRoot = ./.;
  };
}
```
2024-03-05 12:23:28 +13:00
Lily Foster
8999c9d930 npmHooks.npmInstallHook: ignore rather than error for missing bin/man keys in package.json 2023-12-24 22:58:32 +01:00
Lily Foster
b5ca84b450 npmHooks.npmInstallHook: only overwrite npm cache for npm pack rather than for entire hook 2023-12-05 21:53:59 +01:00
Lily Foster
63fabdebd0
Merge pull request #267912 from reinismu/fetch-npm-deps-fix-bug
prefetch-npm-deps: add support for npm alias schema in version spec
2023-12-05 14:39:04 -05:00
Janik
b678419eac
Merge pull request #269078 from lilyinstarlight/fix/prefetch-npm-deps-more-bad-lockfiles 2023-12-03 06:53:59 +01:00
Reinis Muiznieks
02dd7c7bb3 prefetch-npm-deps: add support for npm alias schema in version spec 2023-11-29 13:14:59 +02:00
Lily Foster
25596bd419
Merge pull request #257337 from lorenzleutgeb/yarn-file
prefetch-yarn-deps: Fix handling of scoped packages
2023-11-26 16:49:33 -05:00
Lily Foster
09081aa859
fetchNpmDeps: add test case where empty default lockfile packages is needed 2023-11-23 20:52:00 -05:00
Lily Foster
77571a847f
prefetch-npm-deps: use default value when lockfile has no deps 2023-11-23 20:51:59 -05:00
Lily Foster
daec4bf734
prefetch-npm-deps: instrument some logging 2023-11-23 20:51:58 -05:00
Lily Foster
81ed58b0fe
prefetch-npm-deps: make cargo happy 2023-11-23 20:51:22 -05:00
Lily Foster
ba656ad84e
prefetch-npm-deps: bump deps 2023-11-23 20:51:14 -05:00
Nick Cao
a68bc4feaf
Merge pull request #266296 from yayayayaka/element-1.11.48
element-{web,desktop}: 1.11.47 -> 1.11.50
2023-11-23 09:27:14 -05:00
Janik
8e2c9abca8
Merge pull request #243458 from lilyinstarlight/fix/prefetch-npm-deps-error-when-no-cached-deps 2023-11-23 13:55:03 +01:00
Sandro Jäckel
f16843cb31
fetch-yarn-deps: fix missing cert when fetching packages
Found while updating element-web
2023-11-22 21:25:36 +01:00
Lily Foster
8e3009d95c
buildNpmPackage: add forceEmptyCache option 2023-11-22 15:18:49 -05:00
Lily Foster
05dc145e80
fetchNpmDeps: add forceEmptyCache option 2023-11-22 15:18:39 -05:00
Lily Foster
ec51a56dfc
prefetch-npm-deps: detect and error out when generating an empty cache 2023-11-22 15:18:16 -05:00
Lorenz Leutgeb
29cf6a0422 prefetch-yarn-deps: Fix access to .resolved
... and simplify `prefetchYarnDeps`
2023-11-22 18:03:48 +01:00
Lorenz Leutgeb
3843224a55 prefetch-yarn-deps: Fix parsing of scoped packages 2023-11-22 18:03:06 +01:00
Lily Foster
ab99231a36
prefetch-yarn-deps: add cacert to provide certificates during fetches
Ideally fetch-yarn-deps could do like some other fetchers and support
using SSL_CERT_FILE if it exists and also only verify integrity on FOD
hash unless using an empty/test hash.

But this should keep at least the same semantics as before the recent
Node.js change to stop using the built-in certificate store in favor of
the system one (which does not exist by default in the build sandbox).
2023-11-21 16:24:57 -05:00
midchildan
79459354cb buildNpmPackages: add missing deependencies 2023-11-17 07:58:58 +00:00
Felix Bühler
05a19f48a9
Merge pull request #267778 from lilyinstarlight/fix/yay-more-fixup-yarn-lock-stuff
prefetch-yarn-deps: re-add git hash for fixup
2023-11-16 21:06:38 +01:00
happysalada
7fb490674f buildNpmPackage: allow passing npmDeps 2023-11-16 11:57:48 +00:00
Lily Foster
d02eb2d0e5
prefetch-yarn-deps: re-add git hash for fixup 2023-11-15 21:02:52 -05:00
Janik
582f1e7321
Merge pull request #254420 from lilyinstarlight/fix/npm-pack-pls-😭
npmHooks.npmInstallHook: avoid script output in npm pack command
2023-11-13 22:43:57 +01:00
Yt
5aaeafbe26
buildNpmPackage: make nodejs overridable (#265171)
* buildNpmPackage: allow nodejs to be passed as argument

* Update doc/languages-frameworks/javascript.section.md

Co-authored-by: Lily Foster <lily@lily.flowers>

---------

Co-authored-by: Lily Foster <lily@lily.flowers>
2023-11-09 16:19:24 +00:00
Weijia Wang
093f098d26
Merge pull request #260011 from lilyinstarlight/fix/prefetch-npm-deps-network-error-recovery
prefetch-npm-deps: read url bodies within the retry loop
2023-10-12 19:58:20 +02:00
Lily Foster
554e2412e0
prefetch-npm-deps: read url bodies within the retry loop 2023-10-10 19:40:43 -04:00