ruby.withPackages: init

Co-authored-by: Alyssa Ross <hi@alyssa.is>

doc/languages-frameworks/ruby.section.md

@@ -0,0 +1,365 @@
+---
+title: Ruby
+author: Michael Fellinger
+date: 2019-05-23
+---
+
+# Ruby
+
+## User Guide
+
+### Using Ruby
+
+#### Overview
+
+Several versions of Ruby interpreters are available on Nix, as well as over 250 gems and many applications written in Ruby.
+The attribute `ruby` refers to the default Ruby interpreter, which is currently
+MRI 2.5. It's also possible to refer to specific versions, e.g. `ruby_2_6`, `jruby`, or `mruby`.
+
+In the nixpkgs tree, Ruby packages can be found throughout, depending on what
+they do, and are called from the main package set. Ruby gems, however are
+separate sets, and there's one default set for each interpreter (currently MRI
+only).
+
+There are two main approaches for using Ruby with gems.
+One is to use a specifically locked `Gemfile` for an application that has very strict dependencies.
+The other is to depend on the common gems, which we'll explain further down, and
+rely on them being updated regularly.
+
+The interpreters have common attributes, namely `gems`, and `withPackages`. So
+you can refer to `ruby.gems.nokogiri`, or `ruby_2_5.gems.nokogiri` to get the
+Nokogiri gem already compiled and ready to use.
+
+Since not all gems have executables like `nokogiri`, it's usually more
+convenient to use the `withPackages` function like this:
+`ruby.withPackages (p: with p; [ nokogiri ])`. This will also make sure that the
+Ruby in your environment will be able to find the gem and it can be used in your
+Ruby code (for example via `ruby` or `irb` executables) via `require "nokogiri"`
+as usual.
+
+#### Temporary Ruby environment with `nix-shell`
+
+Rather than having a single Ruby environment shared by all Ruby
+development projects on a system, Nix allows you to create separate
+environments per project.  `nix-shell` gives you the possibility to
+temporarily load another environment akin to a combined `chruby` or
+`rvm` and `bundle exec`.
+
+There are two methods for loading a shell with Ruby packages. The first and
+recommended method is to create an environment with `ruby.withPackages` and load
+that.
+
+```shell
+nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])"
+```
+
+The other method, which is not recommended, is to create an environment and list
+all the packages directly.
+
+```shell
+nix-shell -p ruby.gems.nokogiri ruby.gems.pry
+```
+
+Again, it's possible to launch the interpreter from the shell. The Ruby
+interpreter has the attribute `gems` which contains all Ruby gems for that
+specific interpreter.
+
+##### Load environment from `.nix` expression
+
+As explained in the Nix manual, `nix-shell` can also load an expression from a
+`.nix` file. Say we want to have Ruby 2.5, `nokogori`, and `pry`. Consider a
+`shell.nix` file with:
+
+```nix
+with import <nixpkgs> {};
+ruby.withPackages (ps: with ps; [ nokogiri pry ])
+```
+
+What's happening here?
+
+1. We begin with importing the Nix Packages collections. `import <nixpkgs>`
+   imports the `<nixpkgs>` function, `{}` calls it and the `with` statement
+   brings all attributes of `nixpkgs` in the local scope. These attributes form
+   the main package set.
+2. Then we create a Ruby environment with the `withPackages` function.
+3. The `withPackages` function expects us to provide a function as an argument
+   that takes the set of all ruby gems and returns a list of packages to include
+   in the environment. Here, we select the packages `nokogiri` and `pry` from
+   the package set.
+
+##### Execute command with `--run`
+
+A convenient flag for `nix-shell` is `--run`. It executes a command in the
+`nix-shell`. We can e.g. directly open a `pry` REPL:
+
+```shell
+nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "pry"
+```
+
+Or immediately require `nokogiri` in pry:
+
+```shell
+nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "pry -rnokogiri"
+```
+
+Or run a script using this environment:
+
+```shell
+nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "ruby example.rb"
+```
+
+##### Using `nix-shell` as shebang
+
+In fact, for the last case, there is a more convenient method. You can add a
+[shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) to your script
+specifying which dependencies `nix-shell` needs. With the following shebang, you
+can just execute `./example.rb`, and it will run with all dependencies.
+
+```ruby
+#! /usr/bin/env nix-shell
+#! nix-shell -i ruby -p "ruby.withPackages (ps: with ps; [ nokogiri rest-client ])"
+
+require 'nokogiri'
+require 'rest-client'
+
+body = RestClient.get('http://example.com').body
+puts Nokogiri::HTML(body).at('h1').text
+```
+
+### Developing with Ruby
+
+#### Using an existing Gemfile
+
+In most cases, you'll already have a `Gemfile.lock` listing all your dependencies.
+This can be used to generate a `gemset.nix` which is used to fetch the gems and
+combine them into a single environment.
+The reason why you need to have a separate file for this, is that Nix requires
+you to have a checksum for each input to your build.
+Since the `Gemfile.lock` that `bundler` generates doesn't provide us with
+checksums, we have to first download each gem, calculate its SHA256, and store
+it in this separate file.
+
+So the steps from having just a `Gemfile` to a `gemset.nix` are:
+
+```shell
+bundle lock
+bundix
+```
+
+If you already have a `Gemfile.lock`, you can simply run `bundix` and it will
+work the same.
+
+To update the gems in your `Gemfile.lock`, you may use the `bundix -l` flag,
+which will create a new `Gemfile.lock` in case the `Gemfile` has a more recent
+time of modification.
+
+Once the `gemset.nix` is generated, it can be used in a
+`bundlerEnv` derivation. Here is an example you could use for your `shell.nix`:
+
+```nix
+# ...
+let
+  gems = bundlerEnv {
+    name = "gems-for-some-project";
+    gemdir = ./.;
+  };
+in mkShell { buildInputs = [ gems gems.wrappedRuby ]; }
+```
+
+With this file in your directory, you can run `nix-shell` to build and use the gems.
+The important parts here are `bundlerEnv` and `wrappedRuby`.
+
+The `bundlerEnv` is a wrapper over all the gems in your gemset. This means that
+all the `/lib` and `/bin` directories will be available, and the executables of
+all gems (even of indirect dependencies) will end up in your `$PATH`.
+The `wrappedRuby` provides you with all executables that come with Ruby itself,
+but wrapped so they can easily find the gems in your gemset.
+
+One common issue that you might have is that you have Ruby 2.6, but also
+`bundler` in your gemset. That leads to a conflict for `/bin/bundle` and
+`/bin/bundler`. You can resolve this by wrapping either your Ruby or your gems
+in a `lowPrio` call. So in order to give the `bundler` from your gemset
+priority, it would be used like this:
+
+```nix
+# ...
+mkShell { buildInputs = [ gems (lowPrio gems.wrappedRuby) ]; }
+```
+
+
+#### Gem-specific configurations and workarounds
+
+In some cases, especially if the gem has native extensions, you might need to
+modify the way the gem is built.
+
+This is done via a common configuration file that includes all of the
+workarounds for each gem.
+
+This file lives at `/pkgs/development/ruby-modules/gem-config/default.nix`,
+since it already contains a lot of entries, it should be pretty easy to add the
+modifications you need for your needs.
+
+In the meanwhile, or if the modification is for a private gem, you can also add
+the configuration to only your own environment.
+
+Two places that allow this modification are the `ruby` derivation, or `bundlerEnv`.
+
+Here's the `ruby` one:
+
+```nix
+{ pg_version ? "10", pkgs ? import <nixpkgs> { } }:
+let
+  myRuby = pkgs.ruby.override {
+    defaultGemConfig = pkgs.defaultGemConfig // {
+      pg = attrs: {
+        buildFlags =
+        [ "--with-pg-config=${pkgs."postgresql_${pg_version}"}/bin/pg_config" ];
+      };
+    };
+  };
+in myRuby.withPackages (ps: with ps; [ pg ])
+```
+
+And an example with `bundlerEnv`:
+
+```nix
+{ pg_version ? "10", pkgs ? import <nixpkgs> { } }:
+let
+  gems = pkgs.bundlerEnv {
+    name = "gems-for-some-project";
+    gemdir = ./.;
+    gemConfig = pkgs.defaultGemConfig // {
+      pg = attrs: {
+        buildFlags =
+        [ "--with-pg-config=${pkgs."postgresql_${pg_version}"}/bin/pg_config" ];
+      };
+    };
+  };
+in mkShell { buildInputs = [ gems gems.wrappedRuby ]; }
+```
+
+And finally via overlays:
+
+```nix
+{ pg_version ? "10" }:
+let
+  pkgs = import <nixpkgs> {
+    overlays = [
+      (self: super: {
+        defaultGemConfig = super.defaultGemConfig // {
+          pg = attrs: {
+            buildFlags = [
+              "--with-pg-config=${
+                pkgs."postgresql_${pg_version}"
+              }/bin/pg_config"
+            ];
+          };
+        };
+      })
+    ];
+  };
+in pkgs.ruby.withPackages (ps: with ps; [ pg ])
+```
+
+Then we can get whichever postgresql version we desire and the `pg` gem will
+always reference it correctly:
+
+```shell
+$ nix-shell --argstr pg_version 9_4 --run 'ruby -rpg -e "puts PG.library_version"'
+90421
+
+$ nix-shell --run 'ruby -rpg -e "puts PG.library_version"'
+100007
+```
+
+Of course for this use-case one could also use overlays since the configuration
+for `pg` depends on the `postgresql` alias, but for demonstration purposes this
+has to suffice.
+
+#### Adding a gem to the default gemset
+
+Now that you know how to get a working Ruby environment with Nix, it's time to
+go forward and start actually developing with Ruby.
+We will first have a look at how Ruby gems are packaged on Nix. Then, we will
+look at how you can use development mode with your code.
+
+All gems in the standard set are automatically generated from a single
+`Gemfile`. The dependency resolution is done with `bundler` and makes it more
+likely that all gems are compatible to each other.
+
+In order to add a new gem to nixpkgs, you can put it into the
+`/pkgs/development/ruby-modules/with-packages/Gemfile` and run
+`./maintainers/scripts/update-ruby-packages`.
+
+To test that it works, you can then try using the gem with:
+
+```shell
+NIX_PATH=nixpkgs=$PWD nix-shell -p "ruby.withPackages (ps: with ps; [ name-of-your-gem ])"
+```
+
+#### Packaging applications
+
+A common task is to add a ruby executable to nixpkgs, popular examples would be
+`chef`, `jekyll`, or `sass`. A good way to do that is to use the `bundlerApp`
+function, that allows you to make a package that only exposes the listed
+executables, otherwise the package may cause conflicts through common paths like
+`bin/rake` or `bin/bundler` that aren't meant to be used.
+
+The absolute easiest way to do that is to write a
+`Gemfile` along these lines:
+
+```ruby
+source 'https://rubygems.org' do
+  gem 'mdl'
+end
+```
+
+If you want to package a specific version, you can use the standard Gemfile
+syntax for that, e.g. `gem 'mdl', '0.5.0'`, but if you want the latest stable
+version anyway, it's easier to update by simply running the `bundle lock` and
+`bundix` steps again.
+
+Now you can also also make a `default.nix` that looks like this:
+
+```nix
+{ lib, bundlerApp }:
+
+bundlerApp {
+  pname = "mdl";
+  gemdir = ./.;
+  exes = [ "mdl" ];
+}
+```
+
+All that's left to do is to generate the corresponding `Gemfile.lock` and
+`gemset.nix` as described above in the `Using an existing Gemfile` section.
+
+##### Packaging executables that require wrapping
+
+Sometimes your app will depend on other executables at runtime, and tries to
+find it through the `PATH` environment variable.
+
+In this case, you can provide a `postBuild` hook to `bundlerApp` that wraps the
+gem in another script that prefixes the `PATH`.
+
+Of course you could also make a custom `gemConfig` if you know exactly how to
+patch it, but it's usually much easier to maintain with a simple wrapper so the
+patch doesn't have to be adjusted for each version.
+
+Here's another example:
+
+```nix
+{ lib, bundlerApp, makeWrapper, git, gnutar, gzip }:
+
+bundlerApp {
+  pname = "r10k";
+  gemdir = ./.;
+  exes = [ "r10k" ];
+
+  buildInputs = [ makeWrapper ];
+
+  postBuild = ''
+    wrapProgram $out/bin/r10k --prefix PATH : ${lib.makeBinPath [ git gnutar gzip ]}
+  '';
+}
+```

maintainers/scripts/update-ruby-packages

@@ -0,0 +1,13 @@
+#!/usr/bin/env nix-shell
+#!nix-shell -i bash -p bundler bundix
+
+set -euf -o pipefail
+
+(
+  cd pkgs/development/ruby-modules/with-packages
+  rm -f gemset.nix Gemfile.lock
+  bundle lock
+  bundix
+  mv gemset.nix ../../../top-level/ruby-packages.nix
+  rm -f Gemfile.lock
+)

pkgs/development/interpreters/ruby/default.nix

@@ -3,6 +3,7 @@
 , zlib, openssl, gdbm, ncurses, readline, groff, libyaml, libffi, autoreconfHook, bison
 , autoconf, libiconv, libobjc, libunwind, Foundation
 , buildEnv, bundler, bundix
+, makeWrapper, buildRubyGem, defaultGemConfig
 } @ args:
 
 let
@@ -44,6 +45,7 @@ let
       , autoreconfHook, bison, autoconf
       , buildEnv, bundler, bundix
       , libiconv, libobjc, libunwind, Foundation
+      , makeWrapper, buildRubyGem, defaultGemConfig
       }:
       stdenv.mkDerivation rec {
         pname = "ruby";
@@ -195,6 +197,12 @@ let
             ruby = self;
           };
 
+          inherit (import ../../ruby-modules/with-packages {
+            inherit lib stdenv makeWrapper buildRubyGem buildEnv;
+            gemConfig = defaultGemConfig;
+            ruby = self;
+          }) withPackages gems;
+
           # deprecated 2016-09-21
           majorVersion = ver.major;
           minorVersion = ver.minor;

pkgs/development/ruby-modules/gem-config/default.nix

@@ -19,13 +19,13 @@
 
 { lib, fetchurl, writeScript, ruby, kerberos, libxml2, libxslt, python, stdenv, which
 , libiconv, postgresql, v8_3_16_14, clang, sqlite, zlib, imagemagick
-, pkgconfig , ncurses, xapian_1_2_22, gpgme, utillinux, fetchpatch, tzdata, icu, libffi
+, pkgconfig , ncurses, xapian, gpgme, utillinux, fetchpatch, tzdata, icu, libffi
 , cmake, libssh2, openssl, mysql, darwin, git, perl, pcre, gecode_3, curl
 , msgpack, qt59, libsodium, snappy, libossp_uuid, lxc, libpcap, xorg, gtk2, buildRubyGem
 , cairo, re2, rake, gobject-introspection, gdk-pixbuf, zeromq, czmq, graphicsmagick, libcxx
 , file, libvirt, glib, vips, taglib, libopus, linux-pam, libidn, protobuf, fribidi, harfbuzz
-, bison, flex, pango, python3, patchelf
-, libselinux ? null, libsepol ? null
+, bison, flex, pango, python3, patchelf, binutils, freetds, wrapGAppsHook, atk
+, bundler, libsass, libselinux ? null, libsepol ? null
 }@args:
 
 let
@@ -42,8 +42,9 @@ in
 
 {
   atk = attrs: {
-    nativeBuildInputs = [ pkgconfig ];
-    buildInputs = [ gtk2 pcre rake ];
+    dependencies = attrs.dependencies ++ [ "gobject-introspection" ];
+    nativeBuildInputs = [ rake bundler pkgconfig ];
+    propagatedBuildInputs = [ gobject-introspection wrapGAppsHook atk ];
   };
 
   bundler = attrs:
@@ -85,6 +86,38 @@ in
     buildInputs = [ protobuf ];
   };
 
+  cocoapods-acknowledgements = attrs: {
+    dependencies = attrs.dependencies ++ [ "cocoapods" ];
+  };
+
+  cocoapods-deploy = attrs: {
+    dependencies = [ "cocoapods" ];
+  };
+
+  cocoapods-disable-podfile-validations = attrs: {
+    dependencies = [ "cocoapods" ];
+  };
+
+  cocoapods-generate = attrs: {
+    dependencies = attrs.dependencies ++ [ "cocoapods" ];
+  };
+
+  cocoapods-git_url_rewriter = attrs: {
+    dependencies = [ "cocoapods" ];
+  };
+
+  cocoapods-keys = attrs: {
+    dependencies = attrs.dependencies ++ [ "cocoapods" ];
+  };
+
+  cocoapods-open = attrs: {
+    dependencies = [ "cocoapods" ];
+  };
+
+  cocoapods-try-release-fix = attrs: {
+    dependencies = [ "cocoapods" ];
+  };
+
   curb = attrs: {
     buildInputs = [ curl ];
   };
@@ -113,12 +146,13 @@ in
     '';
   };
 
-  fog-dnsimple = attrs: {
-    postInstall = ''
-      cd $(cat $out/nix-support/gem-meta/install-path)
-      rm {$out/bin,bin,../../bin}/{setup,console}
-    '';
-  };
+  fog-dnsimple = attrs:
+    lib.optionalAttrs (lib.versionOlder attrs.version "1.0.1") {
+      postInstall = ''
+        cd $(cat $out/nix-support/gem-meta/install-path)
+        rm {$out/bin,bin,../../bin}/{setup,console}
+      '';
+    };
 
   redis-rack = attrs: {
     dontBuild = false;
@@ -158,8 +192,8 @@ in
   };
 
   gdk_pixbuf2 = attrs: {
-    nativeBuildInputs = [ pkgconfig ];
-    buildInputs = [ rake gdk-pixbuf ];
+    nativeBuildInputs = [ pkgconfig bundler rake ];
+    propagatedBuildInputs = [ gobject-introspection wrapGAppsHook gdk-pixbuf ];
   };
 
   gpgme = attrs: {
@@ -179,8 +213,14 @@ in
   };
 
   gtk2 = attrs: {
-    nativeBuildInputs = [ pkgconfig ] ++ lib.optionals stdenv.isLinux [ utillinux libselinux libsepol ];
-    buildInputs = [
+    nativeBuildInputs = [
+      binutils pkgconfig
+    ] ++ lib.optionals stdenv.isLinux [
+      utillinux libselinux libsepol
+    ];
+    propagatedBuildInputs = [
+      atk
+      gdk-pixbuf
       fribidi
       gobject-introspection
       gtk2
@@ -194,8 +234,8 @@ in
   };
 
   gobject-introspection = attrs: {
-    nativeBuildInputs = [ pkgconfig ];
-    buildInputs = [ gobject-introspection gtk2 pcre ];
+    nativeBuildInputs = [ pkgconfig pcre ];
+    propagatedBuildInputs = [ gobject-introspection wrapGAppsHook glib ];
   };
 
   grpc = attrs: {
@@ -239,6 +279,10 @@ in
     buildFlags = [ "--with-system-v8=true" ];
   };
 
+  execjs = attrs: {
+    propagatedBuildInputs = [ v8 ];
+  };
+
   libxml-ruby = attrs: {
     buildFlags = [
       "--with-xml2-lib=${libxml2.out}/lib"
@@ -333,16 +377,15 @@ in
   };
 
   pango = attrs: {
-    nativeBuildInputs = [ pkgconfig ];
-    buildInputs = [
+    nativeBuildInputs = [
+      pkgconfig
       fribidi
-      gobject-introspection
-      gtk2
       harfbuzz
       pcre
       xorg.libpthreadstubs
       xorg.libXdmcp
     ];
+    propagatedBuildInputs = [ gobject-introspection wrapGAppsHook gtk2 ];
   };
 
   patron = attrs: {
@@ -380,7 +423,12 @@ in
         "
       '';
     } else {
-      buildInputs = [ libsodium ];
+      dontBuild = false;
+      postPatch = ''
+        substituteInPlace lib/rbnacl/sodium.rb \
+          --replace 'ffi_lib ["sodium"' \
+                    'ffi_lib ["${libsodium}/lib/libsodium${stdenv.hostPlatform.extensions.sharedLibrary}"'
+      '';
     };
 
   re2 = attrs: {
@@ -439,6 +487,12 @@ in
 
   sassc = attrs: {
     nativeBuildInputs = [ rake ];
+    dontBuild = false;
+    SASS_LIBSASS_PATH = "${libsass}";
+    postPatch = ''
+      substituteInPlace lib/sassc/native.rb \
+        --replace 'gem_root = spec.gem_dir' 'gem_root = File.join(__dir__, "../../")'
+    '';
   };
 
   scrypt = attrs:
@@ -471,7 +525,7 @@ in
   sup = attrs: {
     dontBuild = false;
     # prevent sup from trying to dynamically install `xapian-ruby`.
-    nativeBuildInputs = [ rake ];
+    nativeBuildInputs = [ bundler rake ];
     postPatch = ''
       cp ${./mkrf_conf_xapian.rb} ext/mkrf_conf_xapian.rb
 
@@ -506,6 +560,7 @@ in
 
   tiny_tds = attrs: {
     nativeBuildInputs = [ pkgconfig openssl ];
+    buildInputs = [ freetds ];
   };
 
   therubyracer = attrs: {
@@ -541,13 +596,13 @@ in
   xapian-ruby = attrs: {
     # use the system xapian
     dontBuild = false;
-    nativeBuildInputs = [ rake pkgconfig ];
-    buildInputs = [ xapian_1_2_22 zlib ];
+    nativeBuildInputs = [ rake pkgconfig bundler ];
+    buildInputs = [ xapian zlib ];
     postPatch = ''
       cp ${./xapian-Rakefile} Rakefile
     '';
     preInstall = ''
-      export XAPIAN_CONFIG=${xapian_1_2_22}/bin/xapian-config
+      export XAPIAN_CONFIG=${xapian}/bin/xapian-config
     '';
   };
 

pkgs/development/ruby-modules/gem/gem-post-build.rb

@@ -7,7 +7,7 @@ ruby = File.join(ENV["ruby"], "bin", RbConfig::CONFIG['ruby_install_name'])
 out = ENV["out"]
 bin_path = File.join(ENV["out"], "bin")
 gem_home = ENV["GEM_HOME"]
-gem_path = ENV["GEM_PATH"].split(File::PATH_SEPARATOR)
+gem_path = ENV["GEM_PATH"].split(File::PATH_SEPARATOR).uniq
 install_path = Dir.glob("#{gem_home}/gems/*").first
 gemspec_path = ARGV[0]
 

pkgs/development/ruby-modules/with-packages/Gemfile

@@ -0,0 +1,159 @@
+source 'https://rubygems.org' do
+  gem 'addressable'
+  gem 'atk'
+  gem 'awesome_print'
+  gem 'bacon'
+  # gem 'bundler' already got a package for that
+  gem 'byebug'
+  gem 'cairo'
+  gem 'cairo-gobject'
+  gem 'camping'
+  # gem 'capybara-webkit' takes too long to build right now since webkit depends on ruby
+  gem 'charlock_holmes'
+  gem 'cld3'
+  gem 'cocoapods'
+  gem 'cocoapods-acknowledgements'
+  gem 'cocoapods-art'
+  gem 'cocoapods-bin'
+  gem 'cocoapods-browser'
+  gem 'cocoapods-bugsnag'
+  gem 'cocoapods-check'
+  gem 'cocoapods-clean'
+  gem 'cocoapods-clean_build_phases_scripts'
+  gem 'cocoapods-core'
+  gem 'cocoapods-coverage'
+  gem 'cocoapods-deintegrate'
+  gem 'cocoapods-dependencies'
+  gem 'cocoapods-deploy'
+  gem 'cocoapods-downloader'
+  gem 'cocoapods-expert-difficulty'
+  gem 'cocoapods-fix-react-native'
+  gem 'cocoapods-generate'
+  gem 'cocoapods-git_url_rewriter'
+  gem 'cocoapods-keys'
+  gem 'cocoapods-no-dev-schemes'
+  gem 'cocoapods-open'
+  gem 'cocoapods-packager'
+  gem 'cocoapods-playgrounds'
+  gem 'cocoapods-plugins'
+  gem 'cocoapods-prune-localizations'
+  gem 'cocoapods-rome'
+  gem 'cocoapods-search'
+  gem 'cocoapods-sorted-search'
+  gem 'cocoapods-static-swift-framework'
+  gem 'cocoapods-stats'
+  gem 'cocoapods-tdfire-binary'
+  gem 'cocoapods-testing'
+  gem 'cocoapods-trunk'
+  gem 'cocoapods-try'
+  gem 'cocoapods-try-release-fix'
+  gem 'cocoapods-update-if-you-dare'
+  gem 'cocoapods-whitelist'
+  gem 'cocoapods-wholemodule'
+  gem 'coderay'
+  gem 'concurrent-ruby'
+  gem 'curb'
+  gem 'curses'
+  gem 'daemons'
+  gem 'dep-selector-libgecode'
+  gem 'digest-sha3'
+  gem 'domain_name'
+  gem 'do_sqlite3'
+  gem 'ethon'
+  gem 'eventmachine'
+  gem 'excon'
+  gem 'faraday'
+  gem 'ffi'
+  gem 'ffi-rzmq-core'
+  gem 'fog-dnsimple'
+  gem 'gdk_pixbuf2'
+  gem 'gio2'
+  gem 'gitlab-markup'
+  gem 'glib2'
+  # gem 'gobject-introspection' fails on require
+  gem 'gpgme'
+  # gem 'grpc' fails to build
+  gem 'gtk2'
+  gem 'hashie'
+  gem 'highline'
+  gem 'hike'
+  gem 'hitimes'
+  gem 'hpricot'
+  gem 'httpclient'
+  gem 'http-cookie'
+  gem 'iconv'
+  gem 'idn-ruby'
+  gem 'jbuilder'
+  gem 'jekyll'
+  gem 'jmespath'
+  gem 'jwt'
+  gem 'libv8'
+  gem 'libxml-ruby'
+  gem 'magic'
+  gem 'markaby'
+  gem 'method_source'
+  gem 'mini_magick'
+  gem 'msgpack'
+  gem 'mysql2'
+  # gem 'mysql' deprecated
+  gem 'ncursesw'
+  gem 'netrc'
+  gem 'net-scp'
+  gem 'net-ssh'
+  gem 'nokogiri'
+  gem 'opus-ruby'
+  gem 'ovirt-engine-sdk'
+  gem 'pango'
+  gem 'patron'
+  gem 'pcaprub'
+  gem 'pg'
+  gem 'pry'
+  gem 'pry-byebug'
+  gem 'pry-doc'
+  gem 'public_suffix'
+  gem 'puma'
+  gem 'rails'
+  gem 'rainbow'
+  # gem 'rbczmq' deprecated
+  gem 'rbnacl'
+  gem 'rb-readline'
+  gem 're2'
+  gem 'redis'
+  gem 'redis-rack'
+  gem 'rest-client'
+  gem 'rmagick'
+  gem 'rpam2'
+  gem 'rspec'
+  gem 'rubocop'
+  gem 'rubocop-performance'
+  gem 'ruby-libvirt'
+  gem 'ruby-lxc'
+  gem 'ruby-progressbar'
+  gem 'ruby-terminfo'
+  gem 'ruby-vips'
+  gem 'rubyzip'
+  gem 'rugged'
+  gem 'sassc'
+  gem 'scrypt'
+  gem 'semian'
+  gem 'sequel'
+  gem 'sequel_pg'
+  gem 'simplecov'
+  gem 'sinatra'
+  gem 'slop'
+  gem 'snappy'
+  gem 'sqlite3'
+  gem 'taglib-ruby'
+  gem 'therubyracer'
+  gem 'thrift'
+  gem 'tilt'
+  gem 'tiny_tds'
+  gem 'treetop'
+  gem 'typhoeus'
+  gem 'tzinfo'
+  gem 'unf_ext'
+  gem 'uuid4r'
+  gem 'whois'
+  # gem 'xapian-ruby' doesn't contain ruby code
+  gem 'zookeeper'
+end

pkgs/development/ruby-modules/with-packages/default.nix

@@ -0,0 +1,75 @@
+{ stdenv, lib, buildEnv, buildRubyGem, ruby, gemConfig, makeWrapper }:
+
+/*
+Example usage:
+nix-shell -E "(import <nixpkgs> {}).ruby.withPackages (pkgs: with pkgs; [ pry nokogiri ])"
+
+You can also use this for writing ruby scripts that run anywhere that has nix
+using a nix-shell shebang:
+  #!/usr/bin/env nix-shell
+  #!nix-shell -i ruby -p "ruby.withPackages (pkgs: with pkgs; [ pry nokogiri ])"
+
+
+Run the following in the nixpkgs root directory to update the ruby-packages.nix:
+./maintainers/scripts/update-ruby-packages
+*/
+
+let
+  functions = import ../bundled-common/functions.nix { inherit lib gemConfig; };
+
+  buildGems = gemset:
+    let
+      realGemset = if builtins.isAttrs gemset then gemset else import gemset;
+      builtGems =
+        lib.mapAttrs (name: initialAttrs:
+          let
+            attrs = functions.applyGemConfigs ({ inherit ruby; gemName = name; } // initialAttrs);
+          in
+            buildRubyGem (functions.composeGemAttrs ruby builtGems name attrs)
+        ) realGemset;
+    in builtGems;
+
+  gems = buildGems (import ../../../top-level/ruby-packages.nix);
+
+  withPackages = selector:
+    let
+      selected = selector gems;
+
+      gemEnv = buildEnv {
+        name = "ruby-gems";
+        paths = selected;
+        pathsToLink = [ "/lib" "/bin" "/nix-support" ];
+      };
+
+      wrappedRuby = stdenv.mkDerivation {
+        name = "wrapped-${ruby.name}";
+        nativeBuildInputs = [ makeWrapper ];
+        buildCommand = ''
+          mkdir -p $out/bin
+          for i in ${ruby}/bin/*; do
+            makeWrapper "$i" $out/bin/$(basename "$i") --set GEM_PATH ${gemEnv}/${ruby.gemPath}
+          done
+        '';
+      };
+
+    in stdenv.mkDerivation {
+      name = "${ruby.name}-with-packages";
+      nativeBuildInputs = [ makeWrapper ];
+      buildInputs = [ selected ruby ];
+
+      unpackPhase = ":";
+
+      installPhase = ''
+        for i in ${ruby}/bin/* ${gemEnv}/bin/*; do
+          rm -f $out/bin/$(basename "$i")
+          makeWrapper "$i" $out/bin/$(basename "$i") --set GEM_PATH ${gemEnv}/${ruby.gemPath}
+        done
+      '';
+
+      passthru = {
+        inherit wrappedRuby;
+        gems = selected;
+      };
+    };
+
+in { inherit withPackages gems buildGems; }

pkgs/development/ruby-modules/with-packages/require_exceptions.nix

@@ -0,0 +1,84 @@
+let
+  cocoapod-plugin = name: ''
+    require "cocoapods"
+    require "#{Gem::Specification.find_by_name(%(${name})).gem_dir}/lib/cocoapods_plugin"
+  '';
+in {
+  actioncable = [ "action_cable" ];
+  actionmailer = [ "action_mailer" ];
+  actionpack = [ "action_pack" ];
+  actionview = [ "action_view" ];
+  activejob = [ "active_job" ];
+  activemodel = [ "active_model" ];
+  activerecord = [ "active_record" ];
+  activestorage = [ "active_storage" ];
+  activesupport = [ "active_support" ];
+  atk = [ "atk" ];
+  CFPropertyList = [ "cfpropertylist" ];
+  cocoapods-acknowledgements = [ "cocoapods" "cocoapods_acknowledgements" ];
+  cocoapods-art = [ "cocoapods_art" ];
+  cocoapods-browser = [ "cocoapods" "cocoapods_plugin" ];
+  cocoapods-bugsnag = cocoapod-plugin "cocoapods-bugsnag";
+  cocoapods-clean = [ "cocoapods_clean" ];
+  cocoapods-coverage = [ "cocoapods_coverage" ];
+  cocoapods-deintegrate = [ ]; # used by cocoapods
+  cocoapods-dependencies = [ "cocoapods_dependencies" ];
+  cocoapods-deploy = cocoapod-plugin "cocoapods-deploy";
+  cocoapods-generate = cocoapod-plugin "cocoapods-generate";
+  cocoapods-git_url_rewriter = cocoapod-plugin "cocoapods-git_url_rewriter";
+  cocoapods-keys = []; # osx only cocoapod-plugin "cocoapods-keys";
+  cocoapods-open = [ "cocoapods" "cocoapods_plugin" ];
+  cocoapods-packager = [ "cocoapods_packager" ];
+  cocoapods-packager-pro = [ ]; # requires osx
+  cocoapods-plugins = [ "cocoapods_plugins" ];
+  cocoapods-sorted-search = [ ]; # requires osx
+  cocoapods-check = cocoapod-plugin "cocoapods-check";
+  cocoapods-disable-podfile-validations = cocoapod-plugin "cocoapods-disable-podfile-validations";
+  cocoapods-stats = [ "cocoapods_stats" ];
+  cocoapods-testing = [ "cocoapods_testing" ];
+  cocoapods-trunk = [ "cocoapods_trunk" ];
+  cocoapods-try = [ "cocoapods_try" ];
+  cocoapods-try-release-fix = cocoapod-plugin "cocoapods-try-release-fix";
+  digest-sha3 = [ "digest/sha3" ];
+  ffi-compiler = [ "ffi-compiler/loader" ];
+  fog-core = [ "fog/core" ];
+  fog-dnsimple = [ "fog/dnsimple" ];
+  fog-json = [ "fog/json" ];
+  forwardable-extended = [ "forwardable/extended" ];
+  gdk_pixbuf2 = [ "gdk_pixbuf2" ];
+  gitlab-markup = [ "github/markup" ];
+  gobject-introspection = [ "gobject-introspection" ];
+  gtk2 = [ ]; # requires display
+  idn-ruby = [ "idn" ];
+  jekyll-sass-converter = []; # tested through jekyll
+  libxml-ruby = [ "libxml" ];
+  multipart-post = [ "multipart_post" ];
+  unicode-display_width = [ "unicode/display_width" ];
+  nap = [ "rest" ];
+  net-scp = [ "net/scp" ];
+  net-ssh = [ "net/ssh" ];
+  nio4r = [ "nio" ];
+  osx_keychain = [ ]; # requires osx
+  ovirt-engine-sdk = [ "ovirtsdk4" ];
+  pango = [ "pango" ];
+  rack-test = [ "rack/test" ];
+  railties = [ "rails" ];
+  rspec-core = [ "rspec/core" ];
+  rspec-expectations = [ "rspec/expectations" ];
+  rspec-mocks = [ "rspec/mocks" ];
+  rspec-support = [ "rspec/support" ];
+  RubyInline = [ "inline" ];
+  ruby-libvirt = [ "libvirt" ];
+  ruby-lxc = [ "lxc" ];
+  ruby-macho = [ "macho" ];
+  ruby-terminfo = [ "terminfo" ];
+  rubyzip = [ "zip" ];
+  sequel_pg = [ "pg" "sequel" "sequel/adapters/postgresql" "sequel_pg" ];
+  simplecov-html = [ ]; # tested through simplecov
+  sinatra = [ "sinatra/base" ];
+  sprockets-rails = [ "sprockets/rails" ];
+  taglib-ruby = [ "taglib" ];
+  websocket-driver = [ "websocket/driver" ];
+  websocket-extensions = [ "websocket/extensions" ];
+  ZenTest = [ "zentest" ];
+}

pkgs/development/ruby-modules/with-packages/test.nix

@@ -0,0 +1,48 @@
+# a generic test suite for all gems for all ruby versions.
+# use via nix-build.
+let
+  pkgs = import ../../../.. {};
+  lib = pkgs.lib;
+  stdenv = pkgs.stdenv;
+
+  rubyVersions = with pkgs; [
+    ruby_2_4
+    ruby_2_5
+    ruby_2_6
+  ];
+
+  gemTests =
+    (lib.mapAttrs
+      (name: gem: [ name ])
+      pkgs.ruby.gems) //
+    (import ./require_exceptions.nix);
+
+  tests = ruby:
+    lib.mapAttrs (name: gem:
+      let
+        test =
+          if builtins.isList gemTests."${name}"
+          then pkgs.writeText "${name}.rb" ''
+                puts "${name} GEM_HOME: #{ENV['GEM_HOME']}"
+                ${lib.concatStringsSep "\n" (map (n: "require '${n}'") gemTests."${name}")}
+              ''
+          else pkgs.writeText "${name}.rb" gemTests."${name}";
+
+        deps = ruby.withPackages (g: [ g."${name}" ]);
+      in stdenv.mkDerivation {
+        name = "test-gem-${ruby.name}-${name}";
+        buildInputs = [ deps ];
+        buildCommand = ''
+          INLINEDIR=$PWD ruby ${test}
+          touch $out
+        '';
+      }
+    ) ruby.gems;
+in
+  stdenv.mkDerivation {
+    name = "test-all-ruby-gems";
+    buildInputs = builtins.foldl' (sum: ruby: sum ++ ( builtins.attrValues (tests ruby) )) [] rubyVersions;
+    buildCommand = ''
+      touch $out
+    '';
+  }

pkgs/development/ruby-modules/with-packages/test.rb

@@ -0,0 +1,76 @@
+#!/usr/bin/env ruby
+
+# this is a quick and dirty test suite for easier analyzing of breakages in a
+# manual testing.
+# For automated testing use the test.nix
+
+require 'fileutils'
+
+class FakeGemfile
+  attr_reader :gems
+
+  def initialize
+    @gems = []
+  end
+
+  def source(_source, &block)
+    instance_exec(&block)
+  end
+
+  def gem(name)
+    @gems << name
+  end
+end
+
+gemfile = File.expand_path(File.join(__dir__, 'Gemfile'))
+packages = FakeGemfile.new.instance_eval(File.read(gemfile), gemfile)
+
+test_cases = packages.map { |pkg| [pkg, "require '#{pkg}'"] }.to_h
+
+test_cases.merge!(
+  'digest-sha3' => "require 'digest/sha3'",
+  'gitlab-markup' => "require 'github/markup'",
+  'idn-ruby' => "require 'idn'",
+  'net-scp' => "require 'net/scp'",
+  'taglib-ruby' => "require 'taglib'",
+  'net-ssh' => "require 'net/ssh'",
+  'ruby-libvirt' => "require 'libvirt'",
+  'ruby-lxc' => "require 'lxc'",
+  'rubyzip' => "require 'zip'",
+  'sinatra' => "require 'sinatra/base'",
+  'libxml-ruby' => "require 'libxml'",
+  'ruby-terminfo' => "require 'terminfo'",
+  'ovirt-engine-sdk' => "require 'ovirtsdk4'",
+  'fog-dnsimple' => "require 'fog/dnsimple'"
+)
+
+test_cases['sequel_pg'] = <<~TEST
+  require 'pg'
+  require 'sequel'
+  require 'sequel/adapters/postgresql'
+  require 'sequel_pg'
+TEST
+
+tmpdir = File.expand_path(File.join(__dir__, 'tests'))
+FileUtils.rm_rf(tmpdir)
+FileUtils.mkdir_p(tmpdir)
+
+failing = test_cases.reject do |name, test_case|
+  test_case = <<~SHELL
+    #!/usr/bin/env nix-shell
+    #!nix-shell -i ruby -E "(import ../../../.. {}).ruby.withPackages (r: [ r.#{name} ] )"
+    #{test_case}
+  SHELL
+
+  file = File.join(tmpdir, "#{name}_test.rb")
+  File.write(file, test_case)
+  FileUtils.chmod('u=wrx', file)
+
+  system(file) && FileUtils.rm(file)
+end
+
+exit if failing.empty?
+
+puts "Following gems failed: #{failing.keys.join(' ')}"
+puts "tests for failing gems remain in #{tmpdir}"
+exit 1

pkgs/top-level/all-packages.nix

@@ -8977,6 +8977,11 @@ in
 
   ruby = ruby_2_5;
 
+  rubyPackages_2_3 = recurseIntoAttrs ruby_2_3.gems;
+  rubyPackages_2_4 = recurseIntoAttrs ruby_2_4.gems;
+  rubyPackages_2_5 = recurseIntoAttrs ruby_2_5.gems;
+  rubyPackages_2_6 = recurseIntoAttrs ruby_2_6.gems;
+
   mruby = callPackage ../development/compilers/mruby { };
 
   scsh = callPackage ../development/interpreters/scsh { };