Commit Graph

48 Commits

Author SHA1 Message Date
figsoda
3f0f0c74e9 lib.generators: simplify toYAML 2022-09-28 16:34:55 -04:00
figsoda
07b3a5a0dd lib/generators.nix: remove unused bindings 2022-09-26 22:18:54 -04:00
Maximilian Bosch
7203788068
lib/generators: withRecursion: don't break attr-sets with special attrs
Closes #168327

The issue reported there can be demonstrated with the following
expression:

    → nix-instantiate --eval -E "with import ./. {}; pkgs.lib.options.showDefs [ { file = \"foo\"; value = pkgs.rust.packages.stable.buildRustPackages; } ]"
    error: attempt to call something which is not a function but a string

           at /home/ma27/Projects/nixpkgs/lib/trivial.nix:442:35:

              441|   isFunction = f: builtins.isFunction f ||
              442|     (f ? __functor && isFunction (f.__functor f));
                 |                                   ^
              443|

Basically, if a `__functor` is in an attribute-set at depth-limit,
`__functor` will be set to `"<unevaluated>"`. This however breaks
`lib.isFunction` which checks for a `__functor` by invoking `__functor`
with `f` itself.

The same issue - "magic" attributes being shadowed by `withRecursion` -
also applies to others such as
`__pretty`/`__functionArgs`/`__toString`.

Since these attributes have a low-risk of causing a stack overflow
(because these are flat attr-sets or even functions), ignoring them in
`withRecursion` seems like a valid solution.
2022-04-12 12:34:23 +02:00
Profpatsch
626ecc7bd7 lib.generators: fix references to test file 2022-03-14 10:30:55 +01:00
Profpatsch
ab03ddc8f4 lib.generators: add toINIWithGlobalSection
As discussed in
https://github.com/NixOS/nixpkgs/pull/118925#issuecomment-821112723,
this is the best way of adding global sections to `toINI` without
employing heuristics (i.e. checking whether something is an attrset).
2022-03-14 10:21:15 +01:00
Linus Heckemann
a3df3d05e5
Merge pull request #131205 from Ma27/showdefs-overflow
lib/modules: improve errors for `options`/`config`-mixups
2021-09-29 11:03:31 +02:00
Maximilian Bosch
681758d415
lib/generators: fix error message 2021-09-28 22:26:51 +02:00
zimbatm
f882fbcee0
lib.generators.toINI: serialize derivations to string
This is the common case when passing a derivation, we want to access the
store path.
2021-09-12 06:26:21 +02:00
zimbatm
9e90a400a5
lib.generators.toGitINI: don't traverse derivations
Consider a derivation a value to be serialized.

    nix-repl> lib.generators.toGitINI { hello = { drv = pkgs.hello; }; }
    error: evaluation aborted with the following error message: 'generators.mkValueStringDefault: attrsets not supported: <derivation /nix/store/533q15q67sl6dl0272dyi7m7w5pwkkjh-hello-2.10.drv>'

Fixes #137390
2021-09-12 06:26:20 +02:00
Maximilian Bosch
5773ae93f7
lib/generators: move limit detection into withRecursion
As suggested in #131205.

Now it's possible to pretty-print a value with `lib.generators` like
this:

    with lib.generators;
    toPretty { }
      (withRecursion { depthLimit = 10; } /* arbitrarily complex value */)

Also, this can be used for any other pretty-printer now if needed.
2021-08-26 00:28:49 +02:00
Maximilian Bosch
55ea29fd8c
lib/generators/toPretty: add evaluation-limit
When having e.g. recursive attr-set, it cannot be printed which is
solved by Nix itself like this:

    $ nix-instantiate --eval -E 'let a.b = 1; a.c = a; in builtins.trace a 1'
    trace: { b = 1; c = <CYCLE>; }
    1

However, `generators.toPretty` tries to evaluate something until it's
done which can result in a spurious `stack-overflow`-error:

    $ nix-instantiate --eval -E 'with import <nixpkgs/lib>; generators.toPretty {  } (mkOption { type = types.str; })'
    error: stack overflow (possible infinite recursion)

Those attr-sets are in fact rather common, one example is shown above, a
`types.<type>`-declaration is such an example. By adding an optional
`depthLimit`-argument, `toPretty` will stop evaluating as soon as the
limit is reached:

    $ nix-instantiate --eval -E 'with import ./Projects/nixpkgs-update-int/lib; generators.toPretty { depthLimit = 2; } (mkOption { type = types.str; })' |xargs -0 echo -e
    "{
      _type = \"option\";
      type = {
        _type = \"option-type\";
        check = <function>;
        deprecationMessage = null;
        description = \"string\";
        emptyValue = { };
        functor = {
          binOp = <unevaluated>;
          name = <unevaluated>;
          payload = <unevaluated>;
          type = <unevaluated>;
          wrapped = <unevaluated>;
        };
        getSubModules = null;
        getSubOptions = <function>;
        merge = <function>;
        name = \"str\";
        nestedTypes = { };
        substSubModules = <function>;
        typeMerge = <function>;
      };
    }"

Optionally, it's also possible to let `toPretty` throw an error if the
limit is exceeded.
2021-08-25 23:18:26 +02:00
Silvan Mosberger
307c0237d7 lib.generators: Handle no drvPath in toPretty
Previously, if a derivation without a `drvPath` was handled, an error
would be thrown:

    nix-repl> lib.generators.toPretty {} { type = "derivation"; }
    error: attribute 'drvPath' missing, at /home/infinisil/src/nixpkgs/lib/generators.nix:251:24

With this commit it doesn't anymore:

    nix-repl> lib.generators.toPretty {} { type = "derivation"; }
    "<derivation ???>"

This matches what `nix repl` outputs:

    nix-repl> { type = "derivation"; }
    «derivation ???»
2021-08-11 15:52:30 +02:00
Emery Hemingway
1cd48efa96 lib/generators: add toDhall 2021-04-01 14:29:57 +02:00
sternenseemann
06d3b28987 Revert "lib/generators: fix toPretty throwing on (partially applied) builtins"
This reverts commit d9a7d03da8.

Reason for this is that it actually doesn't migitate the issue on nix
stable for another reason: builtins.tryEval doesn't prevent the error
generated by builtins.functionArgs from halting evaluation:

> builtins.tryEval (builtins.functionArgs builtins.functionArgs)
error: 'functionArgs' requires a function, at (string):1:19

Thus it seems that there is no workaround to make
lib.generators.toPretty work with nix stable and primops since there is
no way to distinguish between primops and lambdas in nix.
2021-02-01 16:27:38 +01:00
sternenseemann
d9a7d03da8 lib/generators: fix toPretty throwing on (partially applied) builtins
An high level example case of this problem occuring can be found below:

nix-repl> lib.generators.toPretty {} (lib.concatStringsSep "\n")
error: 'functionArgs' requires a function, at /home/lukas/src/nix/nixpkgs/lib/trivial.nix:334:42

However this does not happen on other partially applied functions:

nix-repl> lib.generators.toPretty {} (lib.concatMapStringsSep "\n")
"<function>"

The issue, as it turns out is that while builtins are functions,
builtins.functionArgs throws if is passed a builtin or a partially
applied builtin:

nix-repl> lib.generators.toPretty {} builtins.toString
error: 'functionArgs' requires a function, at /home/lukas/src/nix/nixpkgs/lib/trivial.nix:334:42

nix-repl> lib.generators.toPretty {} (builtins.foldl' (a: b: a + b))
error: 'functionArgs' requires a function, at /home/lukas/src/nix/nixpkgs/lib/trivial.nix:334:42

I'm pretty sure this qualifies as a nix bug and should be filed
accordingly, but we can work around it in lib.generators.toPretty by
using tryEval and falling back to {} which functionArgs _should_ return
for builtins.

The nix behavior is inconsistent to say the least:

nix-repl> builtins.functionArgs builtins.functionArgs
error: 'functionArgs' requires a function, at (string):1:1

nix-repl> builtins.typeOf builtins.functionArgs
"lambda"

builtins.functionArgs (a: 1 + a)
{ }

nix-repl> builtins.typeOf (a: 1 + a)
"lambda"
2021-01-31 15:35:17 +01:00
Silvan Mosberger
15c5ba9d28
lib/generators.toPretty: functors should print as functions
Not attribute sets. So move the function case forward
2020-09-17 18:20:43 +02:00
Silvan Mosberger
05e4d371ef
lib/generators.toPretty: Print [] and {} compactly 2020-09-17 18:20:39 +02:00
Silvan Mosberger
d0be9e9810
lib/generators.toPretty: Switch away from δ and λ
- These symbols can be confusing for those not familiar with them
- There's no harm in making these more obvious
- Terminals may not print them correctly either

Also changes the function argument printing slightly to be more obvious
2020-09-17 18:20:35 +02:00
Silvan Mosberger
073e9b2aed
lib/generators.toPretty: Improved string printing, handling newlines 2020-09-17 18:20:31 +02:00
Silvan Mosberger
47f2eb89c1
lib/generators.toPretty: Implement multiline printing 2020-09-17 18:20:25 +02:00
Silvan Mosberger
4811f54e94
lib/generators.toPretty: Wrap in a go function
As a preparation to the following commit
2020-09-17 18:20:18 +02:00
Silvan Mosberger
0f6231702f
lib/generators.toPretty: Only quote attribute names if necessary 2020-09-17 18:20:08 +02:00
Silvan Mosberger
9df69cba05
lib/generators: Extend mkValueStringDefault with float support 2020-07-29 18:06:34 +02:00
zimbatm
10c96e5b60
lib.generators: add toGitINI
This code was taken from the home-manager project.
2020-03-26 09:28:13 +01:00
Silvan Mosberger
575354babf
lib/generators: Add toINI option for duplicate keys 2020-03-10 16:01:04 +01:00
Profpatsch
18520b7f36 lib/generators: floats are not supported in mkValueStringDefault
They are cut off after a few decimal places; we cannot in good faith
define a default string representation with that.
2020-01-23 14:47:38 +01:00
Daniel Schaefer
786f02f7a4 treewide: Remove usage of isNull
isNull "is deprecated; just write e == null instead" says the Nix manual
2019-04-29 14:05:50 +02:00
Léo Gaspard
455e0ed885 generators: make toPretty handle floats correctly 2018-10-15 14:03:16 +02:00
volth
87f5930c3f [bot]: remove unreferenced code 2018-07-20 18:48:37 +00:00
Matthew Bauer
dc72e8ac06 lib.generators.toPlist: add floats
Nix now supports floats & we can pretty easily map them to Plist’s
<real></real> type. Note that I am unsure how this affects older
version of Nix that may or may not have builtins.isFloat available.
Make sure this satisfies minver.nix’s "1.11" requirement.
2018-07-03 17:14:00 -04:00
Matthew Bauer
337b58950b generators: refactor toPlist
Address PR comments

Refactors

- Rename toPLIST -> toPlist
2018-06-28 15:24:12 -04:00
Matthew Bauer
d361371d23 generators: refactor toPLIST 2018-06-28 11:11:19 -04:00
Matthew Bauer
3210dd3039 generators: add PLIST handling
/cc @LnL7 @3noch
2018-06-27 15:35:07 -04:00
Profpatsch
c84dad316a lib/generators: print paths without quotes & move function down 2018-04-25 15:31:17 +02:00
Profpatsch
fa71407f36 lib/generators: introduce a sane default for mkValueString
So far, `mkValueString` defaulted to `toString`,
which is a bad match for most configuration file formats,
especially because how booleans are formatted.
This also improves error messages for unsupported types.

Add a test to codify the formatting.
2018-03-29 16:53:06 +02:00
Profpatsch
a7e45fdd8e lib/generators: improve documentation a bit 2018-03-29 16:53:06 +02:00
Profpatsch
219ba583b2 lib/generators: improve toPretty
* properly escape strings
* remove one check for booleans
* improve error message
2018-03-29 16:53:06 +02:00
Shea Levy
943592f698
Add setFunctionArgs lib function.
Among other things, this will allow *2nix tools to output plain data
while still being composable with the traditional
callPackage/.override interfaces.
2018-01-31 14:02:19 -05:00
Profpatsch
319722113e lib/generators: abort on pattern match failure 2018-01-03 23:13:50 +01:00
Profpatsch
8e00dcd242 lib/generators: add mkValueString to mkKeyValueDefault generator
This means the generation of values can now be influenced, even down from e.g.
an INI generator.
Breaks the interface of `mkKeyValueDefault` to match its interface to other
generator functions. It might me sensible to rename `mkKeyValue` and
`mkKeyValueSet` to conform to the `toX`-style of generator functions.
2017-11-22 21:53:59 +01:00
Graham Christensen
152c63c9ff
Convert libs to a fixed-point
This does break the API of being able to import any lib file and get
its libs, however I'm not sure people did this.

I made this while exploring being able to swap out docFn with a stub
in #2305, to avoid functor performance problems. I don't know if that
is going to move forward (or if it is a problem or not,) but after
doing all this work figured I'd put it up anyway :)

Two notable advantages to this approach:

1. when a lib inherits another lib's functions, it doesn't
   automatically get put in to the scope of lib
2. when a lib implements a new obscure functions, it doesn't
   automatically get put in to the scope of lib

Using the test script (later in this commit) I got the following diff
on the API:

  + diff master fixed-lib
  11764a11765,11766
  > .types.defaultFunctor
  > .types.defaultTypeMerge
  11774a11777,11778
  > .types.isOptionType
  > .types.isType
  11781a11786
  > .types.mkOptionType
  11788a11794
  > .types.setType
  11795a11802
  > .types.types

This means that this commit _adds_ to the API, however I can't find a
way to fix these last remaining discrepancies. At least none are
_removed_.

Test script (run with nix-repl in the PATH):

  #!/bin/sh

  set -eux

  repl() {
      suff=${1:-}
      echo "(import ./lib)$suff" \
          | nix-repl 2>&1
  }

  attrs_to_check() {
      repl "${1:-}" \
          | tr ';'  $'\n' \
          | grep "\.\.\." \
          | cut -d' ' -f2 \
          | sed -e "s/^/${1:-}./" \
          | sort
  }

  summ() {
      repl "${1:-}" \
          | tr ' ' $'\n' \
          | sort \
          | uniq
  }

  deep_summ() {
      suff="${1:-}"
      depth="${2:-4}"
      depth=$((depth - 1))
      summ "$suff"

      for attr in $(attrs_to_check "$suff" | grep -v "types.types"); do
          if [ $depth -eq 0 ]; then
              summ "$attr" | sed -e "s/^/$attr./"
          else
              deep_summ "$attr" "$depth" | sed -e "s/^/$attr./"
          fi
      done
  }

  (
      cd nixpkgs

      #git add .
      #git commit -m "Auto-commit, sorry" || true
      git checkout fixed-lib
      deep_summ > ../fixed-lib
      git checkout master
      deep_summ > ../master
  )

  if diff master fixed-lib; then
      echo "SHALLOW MATCH!"
  fi

  (
      cd nixpkgs
      git checkout fixed-lib
      repl .types
  )
2017-09-16 21:36:43 -04:00
Profpatsch
5cbc6ca9bb lib/generators: put more information in toPretty lambdas
With `builtins.functionArgs` we can get some information if the first argument
is an attrset and whether the contained fields have default values. Encode that
into the pretty-printed lambda.
2017-06-22 00:58:59 +02:00
Profpatsch
b1ffe5e4c0 lib/generators: toPretty
`toPretty` implements a pretty printer for nix values.
2017-06-22 00:58:59 +02:00
Profpatsch
271c0f5451 lib/generators: mkKeyValueLine -> mkKeyValueLine (#20920)
Rename, since the previous name was potentially confusing.
2016-12-06 23:19:34 +01:00
Profpatsch
ea412cd5a1 lib/generators: add toKeyValue & mkKeyValueLine (#20903)
generators for the common use case of simple config files which hold keys and
values. Used in the implementation for toINI.
2016-12-04 22:11:24 +01:00
Profpatsch
53fc7b8272 lib/generators: add manual documentation
Restructures the functions reference a bit.
2016-11-17 23:19:28 +01:00
Profpatsch
26eb10e771 lib: add generator functions for toJSON & toYAML
They both reference the toJSON builtin, so we get semantic identifiers
that express the intent of the generation.
Both should be able to map each nix value (minus functions) to the
destination config files.

Includes two invocation unit tests.
2016-11-17 23:16:24 +01:00
Profpatsch
61311665cb lib: add ini configuration generator
Many configurations are INI-style files. Attribute sets can be mapped
rather painlessly to the INI format.
This adds a function toINI inside a new generators library section.
Also, unit tests for the default values are provided.
2016-11-17 19:46:41 +01:00