Commit Graph

613 Commits

Author SHA1 Message Date
DavHau
4f35c7b633 rust translator/builder to use dream-lock packages 2022-02-10 11:43:14 +07:00
DavHau
f634038bbf WIP - adapt code to dream lock changes 2022-02-10 11:43:14 +07:00
DavHau
28c29947cd dream-lock structure changes
remove fields:
 - translator
 - translatedBy
 - mainPackageName
 - mainPackageVersion
add fields:
 - defaultPackage
 - packages
2022-02-10 11:43:14 +07:00
DavHau
ea3d1a2ac1 simplify example flake.nix
fixes a bug where a source tree input was interpreted as a laoded dream-lock
2022-02-09 15:13:49 +07:00
DavHau
817f4d3ba0 package-json transl.: add arg 'npmArgs' 2022-02-04 13:40:56 +07:00
Yusuf Bera Ertan
405e63e941
feat: throw an error in getSourceSpec if the requested dep doesnt exist 2022-01-16 13:00:53 +03:00
Yusuf Bera Ertan
2477a55fbb
fix(rust): put relative paths instead of store paths in lockfile for path deps 2022-01-16 12:56:54 +03:00
Yusuf Bera Ertan
e8c4baf602
feat(rust): use script instead of nix code to find crate paths 2022-01-16 12:56:54 +03:00
Yusuf Bera Ertan
3c6af2acbf
feat(rust): remove git deps from subsystem attrs, use getSourceSpec instead 2022-01-16 12:56:54 +03:00
Yusuf Bera Ertan
c9ea0f7c13
feat: add getSourceSpec 2022-01-16 12:56:54 +03:00
Yusuf Bera Ertan
1d8fc17b93
feat(rust): extend the dream.lock.example.json for the subsystem 2022-01-16 12:56:54 +03:00
Yusuf Bera Ertan
5d18abcdab
fix(rust): write cargo-checksum for vendored git dependencies
fix(rust): copy sources instead of symlinking so we can write cargo checksum

feat(rust): add trace for is git

feat(rust): add trace for is git

feat(rust): add trace for git deps

feat(rust): add trace for git deps

fix(rust): use findFirst not findSingle for finding git dep

fix(rust): remove is git trace

refactor(rust): remove unused code
2022-01-16 12:56:54 +03:00
Yusuf Bera Ertan
901f8a4fc8
fix(rust): pass git dependency parsed information to builders, generate cargo config for git dependencies
fix(rust): fix infinite recursion caused by same name

fix(rust): split gitDeps, generate vendor entries properly

fix(rust): fix infinite recursion in builder

fix(rust): apply git source type to generated vendor entries after applying replace-with

fix(rust): don't overwrite cargo config with generated entries
2022-01-16 12:56:54 +03:00
Yusuf Bera Ertan
e5e88a03b5
feat(rust): workaround git dependencies that are in a workspace by locating the crate source more accurately
fix(rust): copy over findCratePath to builder

fix(rust): correct getAllFiles usage
2022-01-16 12:56:54 +03:00
Yusuf Bera Ertan
f380a81016
fix(rust): dont use src.name
feat(rust): add trace to builder
2022-01-16 12:56:54 +03:00
Max
2132262d5c yarn-lock: allow empty quoted strings 2022-01-13 19:34:36 +01:00
Tom Bereknyei
d2143bb96d Parse unquoted strings in yarn.lock
Sometimes there will be unquoted strings (eg: latest) in dependecy
positions. They are resolved elsewhere but results in a parse error.
2022-01-10 11:03:52 -05:00
DavHau
e69cb52ab0 nodejs builder+translator improvements 2021-12-22 17:31:37 +01:00
DavHau
792deb6a6b feature: CLI: add --sub-dirs flag to enable recursive package transaltion 2021-12-22 17:28:34 +01:00
DavHau
72e73fa255 bugfix: unable to parse tsconfig with comments 2021-12-19 17:06:44 +01:00
DavHau
013dc86524 improve electron support 2021-12-19 14:39:33 +01:00
DavHau
a08f5eb25a merge dev 2021-12-11 11:12:52 +07:00
DavHau
8934508e66 refactor simpleTranslate 2021-12-11 09:13:26 +07:00
DavHau
6a96f26e14 nodejs builder: support electron apps 2021-12-11 09:07:14 +07:00
DavHau
dafda69911 add.py: imporve handling+naming of subpackages 2021-12-11 09:05:59 +07:00
DavHau
87b6959039 fix updating packages + add test 2021-12-09 15:47:08 +07:00
DavHau
728677b654 nodejs package-json: improve + add arguments 2021-12-09 14:46:12 +07:00
DavHau
ed08f9e24c fix fetchers: npm, pypi 2021-12-09 12:55:32 +07:00
DavHau
d608742406 fix: yarn-lock translator 2021-12-08 23:29:09 +07:00
DavHau
1358e1a303 always use internal overrides 2021-12-08 23:18:21 +07:00
DavHau
12c9f565aa nodejs: use package name set via CLI 2021-12-08 23:18:21 +07:00
DavHau
c441eabbb8 make nodejs configurable 2021-12-08 23:18:21 +07:00
Yusuf Bera Ertan
0e236db189
fix(build-rust-package): use produceDerivation to apply overrides to the resulting package(s) 2021-12-03 07:40:55 +03:00
DavHau
d3a279875e feature: support multiple packages per source 2021-12-02 14:15:40 +07:00
DavHau
133b2d0093 fix shortcuts + pname, version handling 2021-11-30 12:36:49 +07:00
Yusuf Bera Ertan
0903f5ea43
feat: pass version and pname to fetchers (#60)
* feat: pass version and pname to fetchers, adjust schema for new behaviour

* fix: name -> pname

* refactor: move override warning to simpleTranslate, flatten dependencyInfo
2021-11-30 12:20:27 +07:00
DavHau
c5126b2053 feature: installMethod copy 2021-11-29 18:44:30 +07:00
Yusuf Bera Ertan
bfcd5b6f35
feat: cargo lock translator, rust builder (#59) 2021-11-29 18:34:53 +07:00
DavHau
fb42313192 bugfix add.py 2021-11-24 09:44:43 +07:00
DavHau
b8dc44d0f2 feature: translate subdir of source via ?dir=foo 2021-11-24 08:50:05 +07:00
DavHau
9d2d385273 test nix funcs with pytest via nix_ffi.py 2021-11-20 11:34:04 +07:00
DavHau
5ad6855226 fix: aggregated fetching 2021-11-18 12:09:14 +07:00
DavHau
553d29724f bugfixes: fixes #54, fixes #55
- add package vie http URL
- create default.nix for debugging
- package-lock parse `npm:` reference
2021-11-18 11:33:24 +07:00
DavHau
4cf80a1460 add test for aggregated fetching: github source 2021-11-17 17:40:09 +07:00
DavHau
e99487cee9 rename sourcesCombinedHash -> sourcesAggregatedHash 2021-11-17 17:12:02 +07:00
DavHau
97e2fef121 fix & improve aggregated source fetching 2021-11-17 17:09:25 +07:00
DavHau
87157ecc8b fix NIX_PATH impurity in writePureShellScript 2021-11-17 12:18:22 +07:00
DavHau
b4b7435ca3
Merge branch 'dev' into go 2021-11-17 11:28:22 +07:00
DavHau
3b5b9096e1 hercules ci + improvements
- yarn-lock: add nodejs version argument
 - add impure tests
 - add ci.nix for hercules
2021-11-17 11:23:31 +07:00
Akshat Agarwal
ff9e30b242 add go support using gomod2nix
Co-authored-by: DavHau <hsngrmpf+github@gmail.com>
2021-11-16 23:28:05 +05:30
DavHau
ec63d19cf5 add.py: refactor spaghetti into functions 2021-11-16 17:02:22 +07:00
DavHau
1c31d53605 fix: add packages via git URL + params; fetch submodules 2021-11-16 08:50:00 +07:00
DavHau
079ec025dc nodejs builder: fix executing $buildScript 2021-11-15 18:10:22 +07:00
DavHau
f1e34e0e89 builder API; move override logic into produceDerivation
This should simplify writing builders
2021-11-13 17:41:48 +07:00
DavHau
163db67245 package-lock: handle resolved=false 2021-11-12 21:52:57 +07:00
DavHau
3ec6baf2c4 support parsing github:... URLs 2021-11-12 21:52:32 +07:00
DavHau
def2b8b499 package-lock translator: deal with more weirdnes 2021-11-12 14:34:48 +07:00
DavHau
6d2a9fd95e nodejs buider: fix crash on empty string bin field 2021-11-11 19:08:25 +07:00
DavHau
48af5f0ec1 nodejs builder: just exclude cyclic deps 2021-11-10 14:05:51 +07:00
DavHau
6097051a5f automatically add new packages to git 2021-11-10 13:11:33 +07:00
DavHau
9302d24fc3 add name argument to package-lock translator 2021-11-10 13:11:21 +07:00
DavHau
baa442739f add template and docs for contributing builder 2021-11-08 11:47:15 +07:00
DavHau
791bb7638f pure translator template based on simpleTranslate 2021-11-08 09:44:27 +07:00
DavHau
8f743b24d4 yarn-lock: fix parsing github URL with revision 2021-11-07 23:59:46 +07:00
DavHau
9fc0623a48 nodejs builder: improve phases 2021-11-07 15:18:28 +07:00
DavHau
9ff7a0ab58 CLI improvements
- rename command package -> add
  - improve handling of dream2nix config
  - improve purity of cli execution
  - automatically find git repo root
  - improve package update cli
2021-11-07 15:18:28 +07:00
DavHau
60d89f776f improve flake interfaces and apps
- improve exposing apps for other flakes
  - configure overridesDirs via config
2021-11-07 15:18:28 +07:00
DavHau
39db511926 improve override system
- add utils.satisfiesSemver (from poetry2nix)
  - nix-parsec add LICENSE from https://github.com/nprindle/nix-parsec/issues/5
  - remove builtin overrides (moved to dreampkgs)
2021-11-07 15:18:28 +07:00
DavHau
bccf3b25c7 multiple override dirs, nodejs improvements
- nodejs builder: symlink sub dependencies
  - allow multiple directories with overrides
  - rename app: cli -> dream2nix
2021-11-07 15:18:28 +07:00
DavHau
62317e6501 fix: inject 2021-11-07 15:18:28 +07:00
DavHau
74a2a2351e ask to create default.nix by default 2021-11-07 15:18:14 +07:00
DavHau
a936cf44a5 rename dream.lock -> dream-lock.json 2021-11-04 15:48:40 +07:00
DavHau
a1d0ed99a8 restructure / rename dream-lock attributes 2021-11-04 15:36:12 +07:00
DavHau
c2e9d3adf4 yarn: remove noOptional + improve UI 2021-11-04 15:02:44 +07:00
DavHau
b1cb420434 fixes for yarn translator and node builder 2021-11-03 10:46:02 +07:00
DavHau
aa81800846 fixes and improvements for nodejs
- add system specific dream2nix to flake outputs
   - add more overrides
   - overrides allow regex as key
2021-11-02 15:13:34 +07:00
DavHau
f1662dabe5 cleanup package.json translator 2021-11-01 15:16:39 +07:00
DavHau
4ef9af0f73 add translator.package.json
- update default.nix template
  - add npm to dream lock schema
  - handle empty depedency graph in readDreamLock
2021-11-01 14:42:02 +07:00
DavHau
5a00d557b9 add npm fetcher 2021-11-01 13:48:23 +07:00
DavHau
03ef2e4c7a detect cycles in pure nix 2021-11-01 13:30:37 +07:00
DavHau
2fd7782af4 nodejs: enable dev deps by default 2021-11-01 13:30:37 +07:00
DavHau
6850db9cc4 cli: rename --extra-arg -> --arg 2021-11-01 13:30:37 +07:00
DavHau
c120dbe328 allow passing translatorArgs to riseAndShine 2021-11-01 13:30:31 +07:00
DavHau
8189d77a7e fix unpackging tgz with broken permissions 2021-11-01 10:03:10 +07:00
DavHau
b23e9ef8f3 make output packages flakes compatible
- closes #39
2021-11-01 00:14:36 +07:00
DavHau
cd6095406b split nodejs builder into different phases 2021-10-31 21:43:45 +07:00
DavHau
0d1e36f770 init: dream2nix overrides 2021-10-31 10:38:51 +07:00
DavHau
120b2d0ee0 improve overrides:
- make overrides referenceable
  - automatically detect override functions for attributes
2021-10-30 19:23:55 +07:00
DavHau
d20f318b58 rename several things
- rename fetcher 'fetchurl' -> 'http'
  - rename specialArgs -> extraArgs
  - rename order_dict -> sort_dict
2021-10-29 21:18:58 +07:00
DavHau
52740e8f96 Tests + refactor externls
- add flake checks
  - refactor handling of external sources
  - rename 'dependenciesRemoved' -> 'cyclicDependencies'
2021-10-29 21:18:58 +07:00
DavHau
c4b110bc38 output schema flakes-style 2021-10-29 16:59:56 +07:00
DavHau
607be5f99c nodejs: improve builder
- stop using npm install
  - run postinstall scripts
  - make binaries of direct dependencies available during build
2021-10-29 16:21:42 +07:00
Daniel Thwaites
84a2a606cc
fix typo emtpy → empty 2021-10-29 07:37:21 +01:00
DavHau
c5881e2db1 pass nodeDeps as path 2021-10-28 14:23:53 +07:00
DavHau
9d8292878a stop using string substitutions in installPhase 2021-10-27 14:42:57 +07:00
DavHau
9e2b46edfd improve nodejs builder + bug fixes
- add option `dontNpmInstall`
  - handle package incompatible to platform
  - replace version in package.json on mismatch
  - fix generated versions for github revisions
2021-10-27 13:24:36 +07:00
DavHau
1219a10592 overrides: allow passing functions for attributes 2021-10-27 13:24:36 +07:00
DavHau
d6dc1560cc fix update command 2021-10-26 20:06:43 +07:00
DavHau
99a87ea7cd add getCyclicDependencies to dream-lock interface 2021-10-26 19:46:55 +07:00
DavHau
1cbc64fe73 nodejs: never execute scripts on installation 2021-10-26 19:29:19 +07:00
DavHau
f49f05e943 improve nodejs builders and translators
- add buildPackageWithOtherBuilder helper
  - do not delete cyclic dependencies by default
  - always delete devDependencies in package.json
  - fix handling of source type `path`
  - yarn-lock: handle missing name
  - update utils.dreamLock interface
  - add `createMissingSource` capability to simpleTranslator
2021-10-26 15:25:35 +07:00
DavHau
70cf6c2723 new builder interface:
- dream lock interface via utils.readDreamLock
  - improve granular builder for nodejs
  - support fetching urls with sha1 hash
  - add jsonschema for source type path
2021-10-23 18:50:36 +07:00
DavHau
9ac7e462f3 improve granular nodejs builder 2021-10-22 23:10:14 +07:00
DavHau
b79343b398 finalize conditional overrides 2021-10-22 23:05:20 +07:00
DavHau
8a5108a602 fix combined hashing mode 2021-10-22 16:29:29 +07:00
DavHau
72c435b8a8 yarn-lock: use git fetcher instead of github 2021-10-22 15:55:45 +07:00
DavHau
38f1725a24 fix dream-lock-schema.json 2021-10-22 15:52:20 +07:00
DavHau
68f38ac988 remove option allowBuiltinFetchers 2021-10-22 15:52:06 +07:00
DavHau
055102e698 use utils.writePureShellScript for all translators 2021-10-22 14:56:56 +07:00
DavHau
6a4a1da0f1 refactor dream.lock sources structure:
- now accessed like sources.{name}.{version}
2021-10-22 14:29:38 +07:00
DavHau
3cbac60c4d add: support git URLs in package-lock parser 2021-10-21 13:45:49 +07:00
DavHau
39b3851ceb fix: extractSource 2021-10-20 21:36:22 +07:00
DavHau
850dc71568 fix python translator+fetcher+builder 2021-10-20 16:39:11 +07:00
DavHau
bf4b807e34 add: update cli app for automatic package update 2021-10-19 00:23:48 +07:00
DavHau
5083a88e4c rename: npmlock2nix -> package-lock 2021-10-18 00:16:13 +07:00
DavHau
99cee262e5 refactor npmlock2nix to use simpleTranslate 2021-10-18 00:15:54 +07:00
DavHau
0867db4b8a move translator yarn-lock-2 -> yarn-lock 2021-10-17 23:08:08 +07:00
DavHau
35a806264f remove old yarn-lock parser 2021-10-17 23:07:18 +07:00
DavHau
353cc807be refine simpleTranslate 2021-10-17 23:03:55 +07:00
DavHau
9677c12e9e add: simpleTranslate helper function for translators 2021-10-17 15:48:32 +07:00
DavHau
7e00121e33 Merge remote-tracking branch 'raghu/main' into dev 2021-10-13 22:52:25 +07:00
DavHau
8ef88fc523 add justBuild (simplified interface) 2021-10-12 23:16:00 +07:00
DavHau
788134d4e8 cli: add option --default-nix 2021-10-12 22:25:06 +07:00
DavHau
7edf7cbf1d granular nodejs builder 2021-10-09 18:33:45 +07:00
DavHau
f7472d096b WIP: conditional overrides 2021-10-09 18:33:19 +07:00
raghuram
441f10fd64 moved specifications folder 2021-10-08 15:29:45 +00:00
raghuram
0ca35d09ad moved src 2021-10-08 15:12:55 +00:00
DavHau
5be5ae49a5 readDreamLock: revert autmatic top-level packages 2021-10-08 11:38:09 +07:00
DavHau
f9752da027 yarn-lock: add flags dev, optional, peer 2021-10-08 11:37:20 +07:00
DavHau
35f9cec8d0 fix github dependency for yarn v1 2021-10-08 00:43:46 +07:00
DavHau
74bff7571b fix: top-lvl deps for package-lock and yarn-lock 2021-10-07 17:07:52 +07:00
DavHau
d5801d1804 bugfix in parseDreamLock 2021-10-07 17:06:09 +07:00
DavHau
d06519bce0 fix parsing CRLF formatted files 2021-10-07 16:25:28 +07:00
DavHau
1c48002dfc fix bugs 2021-10-07 14:38:19 +07:00
happysalada
fb372a0f51 translators: yarn: add parsing of links 2021-10-07 15:07:05 +09:00
DavHau
40d3a3e46b different improvements:
- templating for default.nix
- templates for translators
- fixup nodejs translators
2021-10-07 12:07:28 +07:00
DavHau
e99e002920 Squashed commit of add_yarn_lock:
commit 3415d7f6f834dc3e52cf6898ec02b58ac7df4089
Merge: fca3973 daa71d3
Author: DavHau <hsngrmpf+github@gmail.com>
Date:   Wed Oct 6 10:33:52 2021 +0700

    Merge remote-tracking branch 'happysalada/add_yarn_lock' into dev

commit daa71d3e53
Author: happysalada <raphael@megzari.com>
Date:   Wed Oct 6 00:01:26 2021 +0900

    yarn.lock: initial translator

commit 8f4632ab5e
Author: happysalada <raphael@megzari.com>
Date:   Tue Oct 5 20:52:00 2021 +0900

    translators: add yarn.lock
2021-10-07 11:52:51 +07:00
DavHau
8264abe9c1 remove old cli entry 2021-10-06 10:16:00 +07:00
DavHau
7f19e0662f finalize new CLI 2021-10-06 10:16:00 +07:00
DavHau
66c1ce3c0d add finders for finding new package versions 2021-10-06 10:16:00 +07:00
DavHau
9adf141700 simplify git shortcut + add versionField 2021-10-06 10:16:00 +07:00
DavHau
aa6c0c7027 introduce fetcher modules 2021-10-06 10:16:00 +07:00
DavHau
15ae842a7b WIP revamp cli using cleo 2021-10-06 10:16:00 +07:00
happysalada
1f3b7713a7 fix typo 2021-10-05 16:37:04 +09:00
DavHau
ed145173f5 fix some strings and comments 2021-09-30 01:35:46 +01:00
DavHau
f3444c90f7 add app 'contribute' to initialize templates 2021-09-24 14:19:48 +01:00
DavHau
8d4cf7565a add templates for adding translators 2021-09-23 11:05:25 +01:00
DavHau
e4b9717fa2 fix node2nix builder 2021-09-22 00:30:56 +01:00
DavHau
35eeba24df feature: remove cyclic dependencies 2021-09-20 22:09:14 +01:00
DavHau
f685e44d59 Add builder for nodejs
- python builder support application
- add version to dream.lock
- allowBuiltinsFetchers config option
- node2nix builder
- handle github sources without hash
2021-09-20 20:52:31 +01:00
DavHau
33c7912037 rename: generic lock -> dream lock 2021-09-18 18:08:08 +01:00
DavHau
aa7902a09e improve translators:
- translator flags
  - translatedBy
  - translatorParams
2021-09-18 01:06:48 +01:00
DavHau
d23fb76ef6 feature: sourceOverrides 2021-09-17 13:12:07 +01:00
DavHau
76ae17fc27 support 'unknown' source type 2021-09-17 12:18:39 +01:00
DavHau
042d203237 feature: special arguments for translators 2021-09-17 10:46:03 +01:00
DavHau
7dccfd6e3b separate handling of input files and input dirs 2021-09-16 19:33:31 +01:00
DavHau
b6dfa1dbd4
feature: autoselect translator (#4) 2021-09-16 15:05:31 +01:00
DavHau
a1a3e57e05 add translator: npmlock2nix 2021-09-14 03:00:29 +01:00
DavHau
d83d76d5ef improve handling of translator modules 2021-09-13 18:08:28 +01:00
DavHau
4a00410a2b add support for opt-in FOD fetching
- will ommit storing individual hashes, reducing lock file size
2021-09-11 00:06:48 +02:00
DavHau
72bbad7d9d minor fixes 2021-09-10 10:15:44 +02:00
DavHau
7ef59b51d5 move translator cli to src/apps 2021-09-10 09:44:45 +02:00
DavHau
2ce1ddb6f3 implement UI and examples:
- generic translator CLI exposed as flake app
 - flake app 'install' to install the framework
 - generic fetcher
 - simple translator for python
 - simple builer for python
2021-09-09 23:42:56 +02:00