daml/BAZEL-haskell.md

Haskell in Bazel

Finding your way around our Bazel build system from a Haskell developers' point of view might seem confusing at first. Going beyond just adding targets to BUILD.bazel files requires a more detailed understanding of the system:

• Where rules come from;
• How toolchains and external dependencies are defined;
• Specifiying stock bazel command options.

For this, one needs awareness of four files at the root level of the Daml repository : WORKSPACE, deps.bzl, BUILD and .bazelrc.

.bazelrc the Bazel configuration file

The bazel command accepts many options. To avoid having to specify them manually for every build they can be collected into a .bazelrc file. The root of daml.git contains such a file. There doesn't seem to be anything in ours that is Haskell specific.

WORKSPACE

The root of daml.git is a Bazel "workspace" : there exists a file WORKSPACE. In short, in a WORKSPACE we declare external packages and register toolchains. Visible in a WORKSPACE are the targets of the BUILD.bazel file at the same level as WORKSPACE and any BUILD.bazel files contained in sub-directories of the directory containing WORKSPACE.

Bazel extensions are loaded by a load statement. More or less the first couple of lines of our WORKSPACE reads:

load("//:deps.bzl", "daml_deps")
daml_deps()

Much of the contents of the WORKSPACE file have been factored out into deps.bzl so that other projects can share the definitions contained there. Looking into deps.bzl it begins:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")

This loads the contents of the files http.bzl and git.bzl from the external workspace bazel_tools into the "environment". bazel_tools is an external workspace builtin to Bazel and provides rules for working with archives and git.

[Note : Confusingly (?), //bazel_tools is a Daml package (a sub-directory of the root package directory containing a BUILD.bazel file). Don't confuse @bazel_tools//.. with //bazel_tools/..].

Straight after the loading of those rules, deps.bzl reads,

http_archive(
  name = "rules_haskell",
  strip_prefix = 'rules_haskell-%s' % rules_haskell_version,
  urls = ["https://github.com/tweag/rules_haskell/archive/%s.tar.gz" % rules_haskell_version],
)

This defines the workspace rules_haskell (we call this "rules_haskell" informally - in short, build rules for Haskell) as an external workspace that is downloaded via http. From here on we can refer to things in that workspace by prefixing them with @rules_haskell as in the next command from WORKSPACE,

load("@rules_haskell//haskell:repositories.bzl", "rules_haskell_dependencies")

which has the effect of making the macro rules_haskell_dependencies available in the environment which provides "all repositories necessary for rules_haskell to function":

rules_haskell_dependencies()

As mentioned earlier, targets of any BUILD.bazel file in a package are visible within WORKSPACE. In fact, its a rule that toolchains can only be defined in BUILD.bazel files and registered in WORKSPACE files. register_toolchains registers a toolchain created with the toolchain rule so that it is available for toolchain resolution.

register_toolchains(
  "//:c2hs-toolchain",
)

Those toolchains are defined in BUILD (we'll skip listing their definitions here).

The GHC toolchain is registered within macros provided by rules_haskell:

haskell_register_ghc_nixpkgs(
    attribute_path = "ghc",
    build_file = "@io_tweag_rules_nixpkgs//nixpkgs:BUILD.pkg",
    compiler_flags = [ ... ],
    ...
    version = "8.6.5",
)
haskell_register_ghc_bindists(
    compiler_flags = common_ghc_flags,
    version = "8.6.5",
) if is_windows else None

On Linux and macOS we import GHC from nixpkgs, while on Windows we download an official bindist.

Rules for importing nix packages are provided in the workspace io_tweag_rules_nixpkgs:

http_archive(
    name = "io_tweag_rules_nixpkgs",
    strip_prefix = "rules_nixpkgs-%s" % rules_nixpkgs_version,
    urls = ["https://github.com/tweag/rules_nixpkgs/archive/%s.tar.gz" % rules_nixpkgs_version],
)
load(
  "@io_tweag_rules_nixpkgs//nixpkgs:nixpkgs.bzl",
  "nixpkgs_local_repository", "nixpkgs_git_repository", "nixpkgs_package", "nixpkgs_cc_configure",
)

nixpkgs_local_repository creates an external repository representing the content of of a Nix package collection, based on Nix expressions stored in files in our //nix directory.

nixpkgs_local_repository(
    name = "nixpkgs",
    nix_file = "//nix:bazel-nixpkgs.nix",
)
nixpkgs_local_repository(
    name = 'dev_env_nix',
    nix_file = '//nix:default.nix',
)

nixpkgs_cc_configure tells Bazel to use compilers and linkers from the Nix package collection for the CC toolchain (overriding auto-detection from the current PATH):

nixpkgs_cc_configure(
    nix_file = "//nix:bazel-cc-toolchain.nix",
    repositories = dev_env_nix_repos,
)

where,

dev_env_nix_repos = {
    "nixpkgs": "@nixpkgs",
    "damlSrc": "@dev_env_nix",
}

Finally, we use the bazel-haskell-deps.bzl file which is loaded from WORKSPACE to define the set of Hackage packages that we want to import into Bazel using the stack_snapshot macro.

stack_snapshot(
    name = "stackage",
    packages = [
        "aeson",
        "aeson-pretty",
        ...
    ],
    vendored_packages = {
        "grpc-haskell-core": "@grpc_haskell_core//:grpc-haskell-core",
        "proto3-suite": "@proto3-suite//:proto3-suite",
    },
    local_snapshot = "//:stack-snapshot.yaml",
    stack_snapshot_json = "//:stackage_snapshot.json",
    flags = {
        "integer-logarithms": ["-integer-gmp"],
        "text": ["integer-simple"],
        ...
    },
    tools = [
        "@alex",
        "@happy",
        ...
    ],
    deps = {
        "digest": ["@com_github_madler_zlib//:libz"],
        "zlib": ["@com_github_madler_zlib//:libz"],
    },

This will generate an external workspace called @stackage that exports all the Hackage packages listed in packages or vendored_packages. We use a custom stack snapshot defined in stack-snapshot.yaml. The items listed in the packages attribute will be fetched using the stack tool as defined in the custom snapshot and will be built using the Cabal library. Additionally, we can provide custom Bazel build definitions for packages using the vendored_packages attribute.

The packages are pinned by the Stackage snapshot, in this case a local_snapshot and in the lock-file defined by stack_snapshot_json. If you wish to update packages, then you need to change the packages and local_snapshot attributes accordingly and afterwards execute the following command on Unix and Windows to update the lock-files:

bazel run @stackage-unpinned//:pin

You can use the ad-hoc Windows machines as described in the release documentation to get access to a Windows machine.

The flags attribute can be used to override default Cabal flags. The tools attribute defines Bazel targets for known Cabal tools, e.g. alex, happy, or c2hs. Finally, the deps attribute can be used to define additional dependencies to individual packages. E.g. the zlib Hackage packages depends on the C library libz.

If you wish to override the version of a package that is fetch from Hackage, or fetch it from a different source such as GitHub at a specific commit, then you should modify the stack-snapshot.yaml file. If, additionally, you wish to patch a package, e.g. to override Cabal version bounds, then you should define a custom Bazel build and add the package to the vendored_packages attribute.

For example, to patch the proto3-suite package add the following snippet to the bazel-haskell-deps.bzl file.

http_archive(
    name = "proto3_suite",
    build_file_content = """
load("@rules_haskell//haskell:cabal.bzl", "haskell_cabal_library")
load("@stackage//:packages.bzl", "packages")
haskell_cabal_library(
    name = "proto3-suite",
    version = "0.4.0.0",
    srcs = glob(["**"]),
    deps = packages["proto3-suite"].deps,
    visibility = ["//visibility:public"],
)
    """,
    patch_args = ["-p1"],
    patches = ["@com_github_digital_asset_daml//bazel_tools:haskell-proto3-suite.patch"],
    sha256 = "6a803b1655824e5bec2c518b39b6def438af26135d631b60c9b70bf3af5f0db2",
    strip_prefix = "proto3-suite-f5ca2bee361d518de5c60b9d05d0f54c5d2f22af",
    urls = ["https://github.com/awakesecurity/proto3-suite/archive/f5ca2bee361d518de5c60b9d05d0f54c5d2f22af.tar.gz"],
)

This will fetch the sources from GitHub at the specified revision and apply the patch located in bazel_tools/haskell-proto3-suite.patch in the daml repository.

BUILD

At the root of the repository, alongside WORKSPACE there exists the top-level package definition file BUILD. The primary purpose of this BUILD file is to define toolchains (but it does a couple of other little things as well).

The directive

package(default_visibility = ["//visibility:public"])

sets the default visibility property globally for our targets as public. This means that our targets can freely be depended upon by other targets.

The load statments

load("@rules_haskell//haskell:defs.bzl",
  "haskell_toolchain", "haskell_toolchain_library",
)
load("@rules_haskell//haskell:c2hs.bzl",
  "c2hs_toolchain",
)

bring the macros haskell_toolchain, haskell_toolchain_library, and c2hs_toolchain into scope from rules_haskell.

haskell_toolchain_library:

• import a package that is prebuilt outside of Bazel

haskell_toolchain:

• declare a GHC compiler toolchain

c2hs_toolchain:

• declare a Haskell c2hs toolchain

Lastly, there are some aliases defined here. For example,

alias(
  name = "damlc",
  actual = "//compiler/damlc"
)

Editor integration

The daml repository is configured to support haskell-language-server with Bazel and the da-hls executable is provided by the dev-env. Take a look at the setup section for example configurations for various editors. haskell-language-server has to be built with the same ghc as the project you're working on. Be sure to either point your editor to the dev-env-provided haskell-language-server by absolute path, or make sure that the dev-env-provided haskell-language-server is in $PATH for your editor.

Note, hls itself is built by Bazel and to load a target into the editor some of its dependencies have to be built by Bazel. This means that start-up may take some time if the required artifacts are not built or cached already.

Also note that the current setup works for modules in the bazel target //compiler/damlc:damlc or in its dependencies. To work on other modules, it should be enough to replace //compiler/damlc:damlc in .hie-bios with the appropriate bazel target and restart the language server.

Further reading:

• "Bazel User Guide" (Daml specific)
• "A Users's Guide to Bazel" (official documentation)
• rules_haskell documentation (core Haskell rules, Haddock support, Linting, Defining toolchains, Support for protocol buffers, Interop with cc_* rules, Workspace rules)