2021-04-19 04:39:35 +03:00
|
|
|
# Nickel
|
|
|
|
|
2021-04-19 13:09:51 +03:00
|
|
|
[![Continuous integration](https://github.com/tweag/nickel/workflows/Continuous%20integration/badge.svg)](https://github.com/tweag/nickel/actions?query=branch%3Amaster)
|
2022-06-09 14:50:03 +03:00
|
|
|
[![Website](https://img.shields.io/website-up-down-green-red/http/cv.lbesson.qc.to.svg)](https://nickel-lang.org)
|
2023-12-20 12:30:58 +03:00
|
|
|
[![Discord](https://img.shields.io/badge/Discord-100000?style=flat&logo=Discord&logoColor=C3C3C3&labelColor=4179DA&color=010101)][discord]
|
2021-04-19 13:09:51 +03:00
|
|
|
|
2021-04-19 04:39:35 +03:00
|
|
|
Nickel is the cheap configuration language.
|
|
|
|
|
|
|
|
Its purpose is to automate the generation of static configuration files - think
|
|
|
|
JSON, YAML, XML, or your favorite data representation language - that are then
|
|
|
|
fed to another system. It is designed to have a simple, well-understood core: it
|
|
|
|
is in essence JSON with functions.
|
|
|
|
|
|
|
|
Nickel's salient traits are:
|
|
|
|
|
|
|
|
- **Lightweight**: Nickel is easy to embed. An interpreter should be simple to
|
|
|
|
implement. The reference interpreter can be called from many programming
|
|
|
|
languages.
|
|
|
|
- **Composable code**: the basic building blocks for computing are functions.
|
|
|
|
They are first-class citizens, which can be passed around, called and
|
|
|
|
composed.
|
2021-04-19 10:29:49 +03:00
|
|
|
- **Composable data**: the basic building blocks for data are records
|
2021-04-19 04:39:35 +03:00
|
|
|
(called *objects* in JSON). In Nickel, records can be merged at will,
|
|
|
|
including associated metadata (documentation, default values, type
|
|
|
|
contracts, etc).
|
2021-04-19 10:38:01 +03:00
|
|
|
- **Typed, but only when it helps**: static types improve code quality, serve as
|
2021-04-19 04:39:35 +03:00
|
|
|
documentation and eliminate bugs early. But application-specific
|
|
|
|
self-contained code will always evaluate to the same value, so type errors
|
|
|
|
will show up at runtime anyway. Some JSON is hard to type. There, types are
|
|
|
|
only a burden. Whereas reusable code - that is, *functions* - is evaluated
|
|
|
|
on potentially infinitely many different inputs, and is impossible to test
|
|
|
|
exhaustively. There, types are precious. Nickel has types, but you get to
|
2021-10-29 03:06:18 +03:00
|
|
|
choose when you want it or not, and it handles safely the interaction between
|
2021-04-19 04:39:35 +03:00
|
|
|
the typed and the untyped world.
|
|
|
|
- **Design by contract**: complementary to the type system, contracts are
|
|
|
|
a principled approach to checking assertions. The interpreter automatically
|
|
|
|
inserts assertions at the boundary between typed and untyped code. Nickel
|
|
|
|
lets users add arbitrary assertions of their own and easily understand why
|
2021-04-19 13:09:51 +03:00
|
|
|
when assertions fail.
|
2020-07-30 14:36:49 +03:00
|
|
|
|
|
|
|
The motto guiding Nickel's design is:
|
|
|
|
> Great defaults, design for extensibility
|
|
|
|
|
2021-04-19 04:39:35 +03:00
|
|
|
There should be a standard, clear path for common things. There should be no
|
|
|
|
arbitrary restrictions that limit what you can do you the one day you need to go
|
|
|
|
beyond.
|
2020-07-30 14:36:49 +03:00
|
|
|
|
|
|
|
## Use cases
|
|
|
|
|
2021-10-29 03:06:18 +03:00
|
|
|
Nickel is a good fit in any situation where you need to generate a complex
|
2021-04-19 04:39:35 +03:00
|
|
|
configuration, be it for a single app, a machine, whole infrastructure, or a
|
2020-07-30 14:36:49 +03:00
|
|
|
build system.
|
|
|
|
|
|
|
|
The motivating use cases are in particular:
|
2022-08-04 18:31:21 +03:00
|
|
|
|
2020-07-30 14:36:49 +03:00
|
|
|
- The [Nix package manager](https://nixos.org/): Nix is a declarative package
|
2021-04-19 04:39:35 +03:00
|
|
|
manager using its own language for specifying packages. Nickel is an
|
|
|
|
evolution of the Nix language, while trying to overcome some of its
|
|
|
|
limitations.
|
|
|
|
- Infrastructure as code: infrastructure is becoming increasingly complex,
|
|
|
|
requiring a rigorous approach to deployment, modification and configuration.
|
|
|
|
This is where a declarative approach also shines, as adopted by
|
|
|
|
[Terraform](https://www.terraform.io/),
|
2020-07-30 14:36:49 +03:00
|
|
|
[NixOps](https://github.com/NixOS/nixops) or
|
|
|
|
[Kubernetes](https://kubernetes.io/), all requiring potentially complex
|
|
|
|
generation of configuration.
|
2021-04-19 04:39:35 +03:00
|
|
|
- Build systems: build systems (like [Bazel](https://bazel.build/)) need
|
|
|
|
a specification of the dependency graph.
|
2020-07-30 14:36:49 +03:00
|
|
|
|
2021-04-19 04:39:35 +03:00
|
|
|
Most aforementioned projects have their own bespoke configuration language. See
|
2022-11-21 19:30:36 +03:00
|
|
|
[Comparison](#comparison). In
|
2021-04-19 04:39:35 +03:00
|
|
|
general, application-specific languages might suffer from feature creep, lack of
|
|
|
|
abstractions or just feel ad hoc. Nickel buys you more for less.
|
2020-07-30 14:36:49 +03:00
|
|
|
|
2023-08-11 19:58:58 +03:00
|
|
|
## The Nickel ecosystem
|
|
|
|
|
|
|
|
Related projects that are part of the Nickel ecosystem:
|
|
|
|
|
|
|
|
- [Terraform-Nickel](https://github.com/tweag/tf-ncl): write Terraform
|
|
|
|
configuration with Nickel
|
2023-10-20 13:09:34 +03:00
|
|
|
- [Organist](https://github.com/nickel-lang/organist): batteries included
|
|
|
|
environments with Nickel inside
|
2023-08-11 19:58:58 +03:00
|
|
|
- [json-schema-to-nickel](https://github.com/nickel-lang/json-schema-to-nickel):
|
|
|
|
generate Nickel contracts from JSON schema specifications.
|
|
|
|
- [rules_nickel](https://github.com/nickel-lang/rules_nickel): generate
|
|
|
|
configuration files using Nickel during a Bazel build
|
|
|
|
- The [nickel-lang](https://github.com/nickel-lang) organization hosts various
|
|
|
|
smaller projects, including a tree-sitter grammar definition for Nickel and
|
|
|
|
editor plugins.
|
|
|
|
|
2020-07-30 14:36:49 +03:00
|
|
|
## Getting started
|
|
|
|
|
2022-06-09 14:50:03 +03:00
|
|
|
Please follow the getting started guide for Nickel users on the [nickel-lang
|
|
|
|
website](https://nickel-lang.org/getting-started). The instructions below are
|
2023-04-13 16:52:59 +03:00
|
|
|
either reproduced for this document to be self-contained or because
|
2022-06-09 14:50:03 +03:00
|
|
|
they are aimed toward hacking on the Nickel interpreter itself (e.g. building
|
2023-06-26 19:58:40 +03:00
|
|
|
the `nickel-lang-core` crate documentation).
|
2022-06-09 14:50:03 +03:00
|
|
|
|
2020-07-30 14:36:49 +03:00
|
|
|
### Run
|
|
|
|
|
2023-04-13 16:52:59 +03:00
|
|
|
1. Get a Nickel binary:
|
|
|
|
- With [flake-enabled](https://nixos.wiki/wiki/Flakes) Nix, run
|
|
|
|
Nickel directly with `nix run github:tweag/nickel`. You can use [our binary
|
|
|
|
cache](https://tweag-nickel.cachix.org/) to prevent rebuilding a lot of
|
|
|
|
packages. Pass arguments to Nickel with an extra `--` as in `nix run
|
|
|
|
github:tweag/nickel -- repl`,
|
|
|
|
- Again with flake-enabled Nix, you can install Nickel in your profile with
|
2024-05-16 18:39:12 +03:00
|
|
|
`nix profile install github:tweag/nickel`. The `nickel` command is then in your
|
2023-04-13 16:52:59 +03:00
|
|
|
`$PATH` and is available anywhere.
|
2023-06-19 20:08:27 +03:00
|
|
|
- If you're running macOS you can use Homebrew to install the Nickel binary
|
|
|
|
with `brew install nickel`.
|
2023-07-11 11:42:33 +03:00
|
|
|
- Without Nix, you can use `cargo run --bin nickel` after [building](#build),
|
|
|
|
passing arguments with an extra `--` as in
|
2024-03-11 11:42:44 +03:00
|
|
|
`cargo run --bin nickel -- eval program.ncl`.
|
2021-04-30 22:43:24 +03:00
|
|
|
|
2020-07-30 14:36:49 +03:00
|
|
|
2. Run your first program:
|
2022-08-04 18:31:21 +03:00
|
|
|
|
|
|
|
```console
|
2023-12-20 12:31:07 +03:00
|
|
|
$ nickel eval <<< '["hello", "world"] |> std.string.join ", "'
|
|
|
|
"hello, world"
|
2022-08-04 18:31:21 +03:00
|
|
|
```
|
|
|
|
|
|
|
|
Or load it from a file:
|
|
|
|
|
|
|
|
```console
|
2023-12-20 12:31:07 +03:00
|
|
|
$ echo 'let s = "world" in "hello, %{s}"' > program.ncl
|
|
|
|
$ nickel eval program.ncl
|
|
|
|
"hello, world"
|
2022-08-04 18:31:21 +03:00
|
|
|
```
|
|
|
|
|
2021-08-10 11:34:49 +03:00
|
|
|
3. Start a REPL:
|
2021-03-29 11:36:54 +03:00
|
|
|
|
2022-08-04 18:31:21 +03:00
|
|
|
```console
|
2023-04-13 16:52:59 +03:00
|
|
|
$ nickel repl
|
2023-12-20 12:31:07 +03:00
|
|
|
nickel> {"hello" = true, "world" = true, "universe" = false}
|
|
|
|
|> std.record.to_array
|
|
|
|
|> std.array.filter (fun {field, value} => value)
|
|
|
|
|> std.array.map (fun {field, value} => field)
|
|
|
|
|> std.string.join ", "
|
2022-08-04 18:31:21 +03:00
|
|
|
|
2023-12-20 12:31:07 +03:00
|
|
|
"hello, world"
|
2022-08-04 18:31:21 +03:00
|
|
|
```
|
2021-08-10 11:34:49 +03:00
|
|
|
|
2022-08-04 18:31:21 +03:00
|
|
|
Use `:help` for a list of available commands.
|
2021-03-29 11:36:54 +03:00
|
|
|
4. Export your configuration to JSON, YAML or TOML:
|
2022-08-04 18:31:21 +03:00
|
|
|
|
2021-08-23 14:56:25 +03:00
|
|
|
```console
|
2023-12-20 12:31:07 +03:00
|
|
|
$ nickel export --format json <<< '{content = "hello, world"}'
|
2021-03-29 11:36:54 +03:00
|
|
|
{
|
2023-12-20 12:31:07 +03:00
|
|
|
"content": "hello, world"
|
2021-03-29 11:36:54 +03:00
|
|
|
}
|
|
|
|
```
|
|
|
|
|
2021-03-31 12:48:57 +03:00
|
|
|
Use `nickel help` for a list of subcommands, and `nickel help <subcommand>`
|
2021-03-29 11:36:54 +03:00
|
|
|
for help about a specific subcommand.
|
2020-07-30 14:36:49 +03:00
|
|
|
|
2023-12-20 12:30:58 +03:00
|
|
|
To get in touch, you can join our
|
|
|
|
[![Discord](https://img.shields.io/badge/Discord-100000?style=flat&logo=Discord&logoColor=C3C3C3&labelColor=4179DA&color=010101)][discord]
|
|
|
|
server.
|
|
|
|
|
2022-01-18 20:22:57 +03:00
|
|
|
#### Editor Setup
|
|
|
|
|
2022-08-04 18:31:21 +03:00
|
|
|
Nickel has syntax highlighting plugins for Vim/Neovim, and VSCode. In-editor
|
|
|
|
diagnostics, type hints, and auto-completion are provided by the Nickel Language
|
|
|
|
Server. Please follow
|
2023-04-13 16:52:59 +03:00
|
|
|
[the LSP guide](https://github.com/tweag/nickel/tree/master/lsp) to set up syntax
|
2022-08-04 18:31:21 +03:00
|
|
|
highlighting and NLS.
|
2022-01-18 20:22:57 +03:00
|
|
|
|
2023-04-13 16:52:59 +03:00
|
|
|
#### Formatting
|
|
|
|
|
2023-12-20 12:31:07 +03:00
|
|
|
To format one or several Nickel source files, use `nickel format`:
|
2023-04-13 16:52:59 +03:00
|
|
|
|
|
|
|
```console
|
2023-12-20 12:31:07 +03:00
|
|
|
nickel format network.ncl container.ncl api.ncl
|
2023-04-13 16:52:59 +03:00
|
|
|
```
|
|
|
|
|
2023-08-11 19:58:58 +03:00
|
|
|
Nickel uses [Topiary](https://github.com/tweag/topiary/) to format Nickel code
|
|
|
|
under the hood.
|
|
|
|
|
2023-04-13 16:52:59 +03:00
|
|
|
Please follow the Formatting Capabilities section of the [LSP
|
|
|
|
documentation](https://github.com/tweag/nickel/tree/master/lsp) to know how to
|
2023-08-11 19:58:58 +03:00
|
|
|
hook up the Nickel LSP and Topiary in order to enable formatting inside your
|
2023-04-13 16:52:59 +03:00
|
|
|
code editor.
|
|
|
|
|
2021-04-30 22:47:41 +03:00
|
|
|
### Build
|
|
|
|
|
|
|
|
[rust-guide]: https://doc.rust-lang.org/cargo/getting-started/installation.html
|
|
|
|
|
|
|
|
1. Download build dependencies:
|
2022-08-04 18:31:21 +03:00
|
|
|
|
2021-04-30 22:47:41 +03:00
|
|
|
- **With Nix**: If you have [Nix](https://nixos.org/nix) installed:
|
2022-08-04 18:31:21 +03:00
|
|
|
|
2021-08-23 14:56:25 +03:00
|
|
|
```console
|
2023-12-20 12:31:07 +03:00
|
|
|
nix-shell # if you don't use Nix flakes
|
|
|
|
nix develop # if you use Nix flakes
|
2021-04-30 22:47:41 +03:00
|
|
|
```
|
2022-08-04 18:31:21 +03:00
|
|
|
|
2023-04-13 16:52:59 +03:00
|
|
|
You will be dropped in a shell, ready to build. You can use
|
2023-02-09 15:44:53 +03:00
|
|
|
[our binary cache](https://tweag-nickel.cachix.org/) to prevent rebuilding
|
|
|
|
a lot of packages.
|
2021-04-30 22:47:41 +03:00
|
|
|
- **Without Nix**: otherwise, follow [this guide][rust-guide] to install Rust
|
|
|
|
and Cargo first.
|
2022-08-04 18:31:21 +03:00
|
|
|
|
2022-01-01 10:07:16 +03:00
|
|
|
1. Build Nickel:
|
2022-08-04 18:31:21 +03:00
|
|
|
|
2021-08-23 14:56:25 +03:00
|
|
|
```console
|
2023-12-20 12:31:07 +03:00
|
|
|
cargo build -p nickel-lang-cli --release
|
2021-04-30 22:47:41 +03:00
|
|
|
```
|
2022-08-04 18:31:21 +03:00
|
|
|
|
2023-04-13 16:52:59 +03:00
|
|
|
And voilà! Generated files are placed in `target/release`.
|
2022-08-04 18:31:21 +03:00
|
|
|
|
2023-12-20 12:31:07 +03:00
|
|
|
You can directly build and run the Nickel binary and pass argument after `--`
|
|
|
|
by using `cargo run`:
|
|
|
|
|
|
|
|
```console
|
|
|
|
cargo run --bin nickel -- eval foo.ncl
|
|
|
|
```
|
|
|
|
|
2023-04-13 16:52:59 +03:00
|
|
|
### Test
|
2021-04-30 22:47:41 +03:00
|
|
|
|
2023-04-13 16:52:59 +03:00
|
|
|
Run tests with
|
2021-04-19 04:39:35 +03:00
|
|
|
|
2021-08-23 14:56:25 +03:00
|
|
|
```console
|
2022-08-04 18:31:21 +03:00
|
|
|
cargo test
|
2020-07-30 14:36:49 +03:00
|
|
|
```
|
|
|
|
|
|
|
|
### Documentation
|
2021-04-19 04:39:35 +03:00
|
|
|
|
2022-06-09 14:50:03 +03:00
|
|
|
The user manual is available [on the nickel-lang.org
|
|
|
|
website](https://nickel-lang.org/user-manual/introduction), and in this
|
|
|
|
repository as a collection of Markdown files in `doc/manual`.
|
|
|
|
|
|
|
|
To get the documentation of the `nickel-lang` codebase itself:
|
|
|
|
|
2020-07-30 14:36:49 +03:00
|
|
|
1. Build the doc:
|
2022-08-04 18:31:21 +03:00
|
|
|
|
|
|
|
```console
|
|
|
|
cargo doc --no-deps
|
|
|
|
```
|
|
|
|
|
2020-07-30 14:36:49 +03:00
|
|
|
2. Open the file `target/doc/nickel/index.html` in your browser.
|
|
|
|
|
|
|
|
### Examples
|
2021-04-19 04:39:35 +03:00
|
|
|
|
2022-03-08 11:27:29 +03:00
|
|
|
You can find examples in the [`./examples`](./examples) directory.
|
|
|
|
|
|
|
|
## Current state and roadmap
|
|
|
|
|
2024-03-13 17:54:30 +03:00
|
|
|
Since version 1.0 released in May 2023, the core design of the language is
|
|
|
|
stable and Nickel is useful for real-world applications. The next steps we plan
|
|
|
|
to work on are:
|
2023-12-20 12:31:07 +03:00
|
|
|
|
2023-05-25 17:25:12 +03:00
|
|
|
The next steps we plan to work on are:
|
2022-03-08 11:27:29 +03:00
|
|
|
|
2022-12-08 12:14:44 +03:00
|
|
|
- Nix integration: being able to seamlessly use Nickel to write packages and
|
2023-10-20 13:09:34 +03:00
|
|
|
shells ([Organist](https://github.com/nickel-lang/organist))
|
2022-12-08 12:14:44 +03:00
|
|
|
- Custom merge functions (second part of the
|
2023-04-18 13:54:12 +03:00
|
|
|
[overriding
|
|
|
|
proposal](https://github.com/tweag/nickel/blob/9fd6e436c0db8f101d4eb26cf97c4993357a7c38/rfcs/001-overriding.md))
|
|
|
|
- Incremental evaluation: design an incremental evaluation model and a caching
|
|
|
|
mechanism in order to perform fast re-evaluation upon small changes to a
|
|
|
|
configuration.
|
2022-03-08 11:27:29 +03:00
|
|
|
- Performance improvements
|
2020-07-30 14:36:49 +03:00
|
|
|
|
2022-11-21 19:30:36 +03:00
|
|
|
## Comparison
|
2021-04-19 04:39:35 +03:00
|
|
|
|
2022-05-21 01:44:59 +03:00
|
|
|
- [CUE](https://cuelang.org/) is a configuration language with a focus on data
|
2021-04-19 04:39:35 +03:00
|
|
|
validation. It introduces a new constraint system backed by a solid theory
|
2020-07-30 14:36:49 +03:00
|
|
|
which ensures strong guarantees about your code. It allows for very elegant
|
|
|
|
schema specifications. In return, the cost to pay is to abandon functions
|
|
|
|
and
|
|
|
|
[Turing-completeness](https://en.wikipedia.org/wiki/Turing_completeness).
|
|
|
|
Nickel's merge system is inspired by the one of CUE, even if since Nickel
|
|
|
|
does have general functions and is Turing-complete, they are necessarily
|
|
|
|
different.
|
|
|
|
- [Nix](https://nixos.org/): The Nix language, or *Nix expressions*, is one of
|
2021-08-10 11:34:49 +03:00
|
|
|
the main inspirations for Nickel. It is a very simple yet powerful lazy
|
2020-07-30 14:36:49 +03:00
|
|
|
functional language. We strive to retain this simplicity, while adding
|
|
|
|
typing capabilities, modularity, and detaching the language from the Nix
|
|
|
|
package manager.
|
|
|
|
- [Dhall](https://dhall-lang.org/) is a statically typed configuration language.
|
|
|
|
It is also inspired by Nix, to which it adds a powerful static type system.
|
2020-08-10 10:34:27 +03:00
|
|
|
However, this forces the programmer to annotate all of their code with types.
|
2020-07-30 14:36:49 +03:00
|
|
|
- [Jsonnet](https://jsonnet.org/) is another language which could be dubbed as
|
|
|
|
"JSON with functions" (and others things as well). It is a lazy functional
|
2023-04-18 13:54:12 +03:00
|
|
|
language with object-oriented features, among which inheritance is similar
|
2020-07-30 14:36:49 +03:00
|
|
|
to Nickel's merge system. One big difference with Nickel is the absence of
|
|
|
|
typing.
|
2023-12-04 13:59:01 +03:00
|
|
|
- [KCL](https://kcl-lang.io/) is a gradually typed configuration language whose validation
|
|
|
|
is based on object-oriented schemas that can be extended through inheritance.
|
|
|
|
Unlike the languages above, its evaluation is strict.
|
2020-07-30 14:36:49 +03:00
|
|
|
- [Pulumi](https://www.pulumi.com/) is not a language in itself, but a cloud
|
|
|
|
tool (like Terraform) where you can use your preferred language for
|
|
|
|
describing your infrastructure. This is a different approach to the problem,
|
|
|
|
with different trade-offs.
|
|
|
|
- [Starlark](https://docs.bazel.build/versions/master/skylark/language.html) is
|
2021-04-19 04:39:35 +03:00
|
|
|
the language of [Bazel](https://bazel.build/), which is a dialect of
|
|
|
|
[Python](https://www.python.org/). It does not have types and recursion is
|
|
|
|
forbidden, making it not Turing-complete.
|
2020-07-30 14:36:49 +03:00
|
|
|
|
2020-08-12 18:56:40 +03:00
|
|
|
See [RATIONALE.md](./RATIONALE.md) for the design rationale and a more detailed
|
2022-11-21 19:30:36 +03:00
|
|
|
comparison with these languages.
|
|
|
|
|
|
|
|
### Comparison with other configuration languages
|
2023-01-23 15:36:04 +03:00
|
|
|
<!-- Intentionally duplicated in `RATIONALE.md`, please update the other one for
|
|
|
|
any change done here -->
|
2022-11-21 19:30:36 +03:00
|
|
|
|
2022-11-21 19:34:27 +03:00
|
|
|
| Language | Typing | Recursion | Evaluation | Side-effects |
|
|
|
|
|----------|-------------------------------|------------|------------|--------------------------------------------------|
|
2022-11-24 12:49:48 +03:00
|
|
|
| Nickel | Gradual (dynamic + static) | Yes | Lazy | Yes (constrained, planned) |
|
2022-11-21 19:34:27 +03:00
|
|
|
| Starlark | Dynamic | No | Strict | No |
|
|
|
|
| Nix | Dynamic | Yes | Lazy | Predefined and specialized to package management |
|
|
|
|
| Dhall | Static (requires annotations) | Restricted | Lazy | No |
|
|
|
|
| CUE | Static (everything is a type) | No | Lazy | No, but allowed in the separated scripting layer |
|
|
|
|
| Jsonnet | Dynamic | Yes | Lazy | No |
|
2023-12-04 13:59:01 +03:00
|
|
|
| KCL | Gradual (dynamic + static) | Yes | Strict | No |
|
2022-11-21 19:34:27 +03:00
|
|
|
| JSON | None | No | Strict | No |
|
|
|
|
| YAML | None | No | N/A | No |
|
|
|
|
| TOML | None | No | N/A | No |
|
2023-12-20 12:30:58 +03:00
|
|
|
|
|
|
|
[discord]: https://discord.gg/vYDnJYBmax
|