add Bazel bash help (#2223)

BAZEL-bash.md

@@ -0,0 +1,148 @@
+# Working with Bash in Bazel
+
+Bazel has two out-of-the-box rules, `sh_test` and `genrule`, that can be used
+for one-shot custom build targets. If there is some reusable logic behind your
+target, consider building a new rule in Skylark instead.
+
+## Platform
+
+There are a number of limitations in the way Bazel will set up the environment
+for these Bash scripts when running them, and they differ based on the OS. The
+instructions given here will hopefully help you get to a point where your rules
+run on all platforms.
+
+## Dependencies
+
+Bazel promises hermeticity, but Bash scripts have access to whatever they want.
+As far as I can tell they are not run in any kind of chroot or container (at
+least on macOS), although they are run with a restricted PATH (on Linux) and in
+a specially set-up folder.
+
+As such, in order to get all of your dependencies available, you need to
+specify them in the relevant attributes:
+- `srcs` for (source) files required during build; note that in the case of
+  `sh_test`, this must be a single-element list that contains just the script
+  you intend to run.
+- `data` for files required at run time (`sh_test` only).
+- `deps` for results of other Bazel rules required at run time.
+- `tools` for executables (`genrule` only).
+
+## `genrule`: substitutions
+
+The code of a `genrule` directive is the `cmd` attribute, which is a multiline
+string embedding a Bash script. Within that script, you need to use `$$` to
+produce a single `$` in the Bash script itself, as `$()` will be expanded by
+Bazel's own string interpolation. You can use the `location` and `locations`
+functions to get paths for Bazel build results. You should note that the paths
+these return is relative to the starting `$PWD` of the script, so if your Bash
+script `cd`s somwhere, you have to take that into account.
+
+You can include local files (files present in your workdir that are not the
+result of another Bazel rule) as part of your `srcs` by using the `glob`
+function, as in:
+```
+    srcs = glob([
+        "package.json",
+        "syntaxes/*",
+    ]) + [
+        "//some:rule",
+    ],
+```
+
+The script will start running at the root of your workspace. While local files
+specified in `srcs` are specified using paths relative to the package root in
+the `glob` directive (i.e. relative to the containing directory of the
+`BUILD.bazel` file); they are added to the run time directory of the script
+using their package-qualified path. If the above snippet came from
+`my/package/BUILD.bazel`, the `package.json` file would be at
+`my/package/package.json`.
+
+Note that tools and data dependencies will also be given as relative (to the
+script's starting directory) paths through the `location` and `locations`
+functions.
+
+## Conditionals
+
+Bazel supports simple conditionals in its rules, though I have not found a way
+to use them in string substitutions. As some rules have different names under
+Windows, this creates a bit of a problem. Aliases provide a way around that;
+for example:
+```
+alias(
+    name = "yarn",
+    actual = "@nodejs//:bin/yarn.cmd" if is_windows else "@nodejs//:bin/yarn",
+)
+```
+would let you use, in the `cmd` entry of a script:
+```
+        $$DIR/$(location :yarn)
+```
+to invoke the correct `yarn` executable regardless of platform.
+
+## Links and speed
+
+Bazel relies a lot on symbolic links to provide what should appear like a new,
+isolated environment to each build rule without having to pay the cost of
+copying files over each time. However, this means that
+1. Everything path-related is different on Windows, because symlinks there are
+   not well-supported, and
+1. While Bazel can claim some nice speed for creating the environment, every
+   read your rule does from a Bazel-provided file takes the hit of going
+   through the symlink indirection.
+That second point can be very costly when you have to read many small files
+(e.g. node dependencies).
+
+## `sh_test`: "runfiles"
+
+Whereas `genrule` takes an inline Bash script as its `cmd` attribute, the
+`sh_test` script will run a Bash script defined in a separate file and
+specified as the single entry in the `srcs` attribute. This adds the
+complication that `location` substitution is not available.
+
+A partial workaround for that is to pass the relevant paths to the script as
+the `args` attribute of the `sh_rule`; those will be available as traditional
+arguments in Bash (`$@`, `$#`, `$1`, etc.).
+
+However, in order for the test script to work under Windows (where symlinks are
+not well supported), another level of indirection is introduced by Bazel: the
+paths returned by the `location` function cannot be used directly by the script
+given to `sh_test`, and must further be translated to "real", absolute paths
+through the use of an `rlocation` Bash function.
+
+To make this function available to your Bash script, the `sh_test` rule must
+include an additional dependency:
+```
+deps = ["@bazel_tools//tools/bash/runfiles"],
+```
+
+Then, anything your script needs access to, including executables, needs to be
+passed in as arguments (using `location` in the `BUILD.bazel` file). In the
+script itself, before using any of the arguments, you have to paste [the
+following snippet from the Bazel runfiles]():
+```
+    # Copy-pasted from the Bazel Bash runfiles library v2.
+    set -uo pipefail; f=bazel_tools/tools/bash/runfiles/runfiles.bash
+    source "${RUNFILES_DIR:-/dev/null}/$f" 2>/dev/null || \
+      source "$(grep -sm1 "^$f " "${RUNFILES_MANIFEST_FILE:-/dev/null}" | cut -f2- -d' ')" 2>/dev/null || \
+      source "$0.runfiles/$f" 2>/dev/null || \
+      source "$(grep -sm1 "^$f " "$0.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \
+      source "$(grep -sm1 "^$f " "$0.exe.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \
+      { echo>&2 "ERROR: cannot find $f"; exit 1; }; f=; set -e
+    # --- end runfiles.bash initialization v2 ---
+```
+After that, in your Bash script, you can construct the absolute paths by taking
+the values Bazel gave you and expanding them like (assuming the first argument
+specified in the `BUILD.bazel` file was a `$(location ...)` call) :
+```
+MY_FIRST_ARG="$(rlocation "$TEST_WORKSPACE/$1")"
+```
+The `TEST_WORKSPACE` variable is set by Bazel to the current WORKSPACE name.
+Note that you should use this name even for dependencies from external
+namespaces.
+
+Note that you should use the above method to access _all_ of your dependencies,
+_including_ "local" files from the same package as the `sh_test`. Trying to
+access those files directly by their expected path, as you can do in
+`genrule`s, will not work with `sh_test` on Windows. You may have to wrap your
+local files in a `filegroup` to have a valid target for the `$(location ...)`
+call you should pass as an `args` to the `sh_test`.