nix-community/dream2nix
1
1
Fork 0
mirror of https://github.com/nix-community/dream2nix.git synced 2024-12-23 14:31:55 +03:00
Code Issues Projects Releases Wiki Activity

docs: improve structure

Browse Source
This commit is contained in:
hsjobeki 2022-10-11 11:37:31 +02:00
parent 9bc8810271
commit 0c8971f1ef
5 changed files with 116 additions and 88 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

30
docs/src/SUMMARY.md
View File

@ -12,13 +12,35 @@
- [Haskell](./subsystems/haskell.md) - [Haskell](./subsystems/haskell.md)
- [PHP](./subsystems/php.md) - [PHP](./subsystems/php.md)
# Concepts # Concepts / API
- [Architectural Considerations](./intro/architectural-considerations.md) - [Architecture](./intro/architecture.md)
- [Nixpkgs improvements](./intro/nixpkgs-improvements.md)
- [Override system](./intro/override-system.md)
- [Translators](./intro/translators.md) - [Translators](./intro/translators.md)
- [npm]()
- [package-json]()
- [package-lock]()
- [yarn-lock]()
- [Generic Lockfile](./api/generic-lock.md)
- [Indexers](./intro/indexers.md) - [Indexers](./intro/indexers.md)
- [pip]()
- [crates-io]()
- [crates-io-simple]()
- [libraries-io]()
- [npm]()
- [Fetchers](./intro/fetchers.md) - [Fetchers](./intro/fetchers.md)
- [git]()
- [github]()
- [gitlab]()
- [http]()
- [path]()
- [npm]()
- [pypi-sdist]()
- [crates-io]()
- [Builders]()
- [nodejs]()
- [Override system](./intro/override-system.md)
# Correlation with nixpkgs
- [Nixpkgs improvements](./intro/nixpkgs-improvements.md)
# Contributing # Contributing
- [Extending dream2nix](./extending-dream2nix.md) - [Extending dream2nix](./extending-dream2nix.md)

36
docs/src/api/generic-lock.md Normal file
View File

@ -0,0 +1,36 @@
# Generic lockfile
The generic lockfile is interpreted from the fetchers and dependencies are loaded based on their type.
When dream2nix reads project dependencies and translates them into
a dream2nix lockfile, it writes down the sources for all the
dependencies: so it can fetch them later during building.
The generic lockfile is used always, either in memory or dumped to a file.
Below are some examples of the sources written in the dream-lock file
(at the end, under `sources`).
```json
"symfony/process": {
"v6.1.3": {
"rev": "a6506e99cfad7059b1ab5cab395854a0a0c21292",
"type": "git",
"url": "https://github.com/symfony/process.git"
}
}
```
```json
"fast-glob": {
"3.2.11": {
"hash": "sha512-xrO3+1bxSo3ZVHAnqzyuewYT6aMFHRAd4Kcs92MAonjwQZLsK9d0SF1IyQ3k5PoirxTW0Oe/RqFgMQ6TcNE5Ew==",
"type": "http",
"url": "https://registry.npmjs.org/fast-glob/-/fast-glob-3.2.11.tgz"
}
}
```
The `type` field tells dream2nix what [fetchers](../intro/fetchers.md) to invoke to get this
source, the rest of the attributes are simply passed to the fetcher.

53
docs/src/intro.md
View File

@ -22,9 +22,60 @@ The goal of this project is to create a standardized, generic, modular framework
The intention is to integrate many existing 2nix converters into this framework, thereby improving many of the previously named aspects and providing a unified UX for all 2nix solutions. The intention is to integrate many existing 2nix converters into this framework, thereby improving many of the previously named aspects and providing a unified UX for all 2nix solutions.
### Modularity:
The following phases which are generic to basically all existing 2nix solutions:
- parsing project metadata
- resolving/locking dependencies (not always required)
- fetching sources
- building/installing packages
... should be separated from each other with well defined interfaces.
This will allow for free composition of different approaches for these phases.
The user should be able to freely choose between:
- input metadata formats (eg. lock file formats)
- metadata fetching/translation strategies: IFD vs. in-tree
- source fetching strategies: granular fetching vs fetching via single large FOD to minimize expression file size
- installation strategies: build dependencies individually vs inside a single derivation.
### Customizability
Every Phase mentioned in the previous section should be customizable at a high degree via override functions. Practical examples:
- Inject extra requirements/dependencies
- fetch sources from alternative locations
- replace or modify sources
- customize the build/installation procedure
### Maintainability
Due to the modular architecture with strict interfaces, contributors can add support for new lock-file formats or new strategies for fetching, building, installing more easily.
### Compatibility
Depending on where the nix code is used, different approaches are desired or discouraged. While IFD might be desired for some out of tree projects to achieve simplified UX, it is strictly prohibited in nixpkgs due to nix/hydra limitations.
All solutions which follow the dream2nix specification will be compatible with both approaches without having to re-invent the tool.
### Code de-duplication
Common problems that apply to many 2nix solutions can be solved once by the framework. Examples:
- handling cyclic dependencies
- handling sources from various origins (http, git, local, ...)
- generate nixpkgs/hydra friendly output (no IFD)
- good user interface
### Code de-duplication in nixpkgs
Essential components like package update scripts or fetching and override logic are provided by the dream2nix framework and are stored only once in the source tree instead of several times.
### Risk free opt-in FOD fetching
Optionally, to save more storage space, individual hashes for source can be omitted and a single large FOD used instead.
Due to a unified minimalistic fetching layer the risk of FOD hash breakages should be very low.
### Common UI across many 2nix solutions
2nix solutions which follow the dream2nix framework will have a unified UI for workflows like project initialization or code generation. This will allow quicker onboarding of new users by providing familiar workflows across different build systems.
### Reduced effort to develop new 2nix solutions
Since the framework already solves common problems and provides an interface for integrating new build systems, developers will have an easier time creating their next 2nix solution.
### Further reading ### Further reading
- [Architectural Considerations](./intro/architectural-considerations.md) - [Architectural Considerations](./intro/architecture.md)
- [Potential impact on nixpkgs](./intro/nixpkgs-improvements.md) - [Potential impact on nixpkgs](./intro/nixpkgs-improvements.md)
[glossary]: https://nixos.wiki/wiki/Glossary "glossary" [glossary]: https://nixos.wiki/wiki/Glossary "glossary"

51
docs/src/intro/architectural-considerations.md → docs/src/intro/architecture.md
View File

@ -1,54 +1,5 @@
### Modularity: # Architecture
The following phases which are generic to basically all existing 2nix solutions:
- parsing project metadata
- resolving/locking dependencies (not always required)
- fetching sources
- building/installing packages
... should be separated from each other with well defined interfaces.
This will allow for free composition of different approaches for these phases.
The user should be able to freely choose between:
- input metadata formats (eg. lock file formats)
- metadata fetching/translation strategies: IFD vs. in-tree
- source fetching strategies: granular fetching vs fetching via single large FOD to minimize expression file size
- installation strategies: build dependencies individually vs inside a single derivation.
### Customizability
Every Phase mentioned in the previous section should be customizable at a high degree via override functions. Practical examples:
- Inject extra requirements/dependencies
- fetch sources from alternative locations
- replace or modify sources
- customize the build/installation procedure
### Maintainability
Due to the modular architecture with strict interfaces, contributors can add support for new lock-file formats or new strategies for fetching, building, installing more easily.
### Compatibility
Depending on where the nix code is used, different approaches are desired or discouraged. While IFD might be desired for some out of tree projects to achieve simplified UX, it is strictly prohibited in nixpkgs due to nix/hydra limitations.
All solutions which follow the dream2nix specification will be compatible with both approaches without having to re-invent the tool.
### Code de-duplication
Common problems that apply to many 2nix solutions can be solved once by the framework. Examples:
- handling cyclic dependencies
- handling sources from various origins (http, git, local, ...)
- generate nixpkgs/hydra friendly output (no IFD)
- good user interface
### Code de-duplication in nixpkgs
Essential components like package update scripts or fetching and override logic are provided by the dream2nix framework and are stored only once in the source tree instead of several times.
### Risk free opt-in FOD fetching
Optionally, to save more storage space, individual hashes for source can be omitted and a single large FOD used instead.
Due to a unified minimalistic fetching layer the risk of FOD hash breakages should be very low.
### Common UI across many 2nix solutions
2nix solutions which follow the dream2nix framework will have a unified UI for workflows like project initialization or code generation. This will allow quicker onboarding of new users by providing familiar workflows across different build systems.
### Reduced effort to develop new 2nix solutions
Since the framework already solves common problems and provides an interface for integrating new build systems, developers will have an easier time creating their next 2nix solution.
### Architecture
The general architecture should consist of these components: The general architecture should consist of these components:
`Input -> Translation -> Generic Lock -> Fetching -> Building` `Input -> Translation -> Generic Lock -> Fetching -> Building`

34
docs/src/intro/fetchers.md
View File

@ -1,38 +1,6 @@
# Fetchers # Fetchers
This section describes the available source code fetchers. This section describes the available source code fetchers. A fetcher is responsible for loading sources defined in the generic lockfile described earlier.
## What are fetchers
When dream2nix reads project dependencies and translates them into
a dream2nix lockfile, it writes down the sources for all the
dependencies: so it can fetch them later during building.
Below are some examples of the sources written in the dream-lock file
(at the end, under `sources`).
```json
"symfony/process": {
"v6.1.3": {
"rev": "a6506e99cfad7059b1ab5cab395854a0a0c21292",
"type": "git",
"url": "https://github.com/symfony/process.git"
}
}
```
```json
"fast-glob": {
"3.2.11": {
"hash": "sha512-xrO3+1bxSo3ZVHAnqzyuewYT6aMFHRAd4Kcs92MAonjwQZLsK9d0SF1IyQ3k5PoirxTW0Oe/RqFgMQ6TcNE5Ew==",
"type": "http",
"url": "https://registry.npmjs.org/fast-glob/-/fast-glob-3.2.11.tgz"
}
}
```
The `type` field tells dream2nix what fetchers to invoke to get this
source, the rest of the attributes are simply passed to the fetcher.
## Fetcher structure ## Fetcher structure