# Bazel User Guide This document explains how to use the Bazel build system on the DAML repository from a users perspective. I.e. assuming the project you are working on has already been ported to Bazel. This guide does not cover how to port a new project to Bazel. Please refer to the [Bazel JVM Porting Guide][bazel_jvm_guide] if you intend to port a JVM project to Bazel. [bazel_jvm_guide]: ./BAZEL-JVM.md ## Setup This section goes through the required steps for a basic but fully functioning setup of the Bazel build system for work on the DAML repository. Additional setup as for the IntelliJ integration is listed in its own section below. ### Bazel Executable Bazel is incorporated in the dev-env. If the [dev-env is setup correctly][dev_env_guide] and `dev-env/bin` is in your `$PATH`, then Bazel should be ready to use. [dev_env_guide]: ./README.md ## Build and Test Once setup is complete, you can build the whole repository with the following command. ``` bazel build //... ``` You can run all hermetic tests in the repository with the following command. ``` bazel test //... ``` ## Bazel Core Concepts If you are unfamiliar with Bazel it is recommended that you read the official [Concepts and Terminology guide][bazel_core_concepts]. Here we will only provide a brief overview which may serve as a refresher. [bazel_core_concepts]: https://docs.bazel.build/versions/master/build-ref.html In short, the `daml` repository is a Bazel *workspace*. It contains a `WORKSPACE` file, which defines external dependencies. The workspace contains several *packages*. A package is a directory that contains a `BUILD.bazel` or `BUILD` file. Each package holds multiple *targets*. Targets are either *files* under the package directory or *rules* defined in the `BUILD.bazel` file. You can address a target by a *label* of the form `//path/to/package:target`. For example, `//ledger/sandbox:sandbox`. Here `sandbox` is a target in the package `ledger/sandbox`. It is defined in the file `ledger/sandbox/BUILD.bazel` using `da_scala_library` as shown below. ``` da_scala_library( name = "sandbox", srcs = glob(["src/main/scala/**/*.scala"]), resources = glob( ["src/main/resources/**/*"], # Do not include logback.xml into the library: let the user # of the sandbox-as-a-library decide how to log. exclude = ["src/main/resources/logback.xml"], ) + [ "//:MVN_VERSION", ], tags = ["maven_coordinates=com.digitalasset.platform:sandbox:__VERSION__"], visibility = [ "//visibility:public", ], runtime_deps = [ "@maven//:ch_qos_logback_logback_classic", "@maven//:ch_qos_logback_logback_core", ], deps = compileDependencies, ) ``` The arguments to `da_scala_library` are called *attributes*. These define the name of the target, the sources it is compiled from, its dependencies, etc. Note, that Bazel build rules are hermetic. I.e. only explicitly declared dependencies will be available during execution. In particular, if a rule depends on additional data files, then these have to be declared dependencies as well. For example using the `resources` or the `data` attributes. The details depend on the rule in question. The following rules are commonly used in this repository. For Scala projects `da_scala_library`, `da_scala_test_suite`, `da_scala_binary`. For Java projects `java_library`, `java_test_suite`, `java_binary`. For DAML projects `daml`. Labels can point to a specific target, or to a set of targets using a wild-card. The following wild-card patterns are recognized. - Ellipsis (`//some/package/...`): All rule targets within or underneath `some/package`. - All (`//some/package:all`): All rule targets within `some/package`. - Star (`//some/package:*`): All rule or file targets within `some/package`. So far we have talked about targets defined in the current workspace. Bazel also has a notion of external workspaces. Targets in external workspaces are labelled as `@workspace_name//path/to/package:target`. Targets have a *visibility* attribute that determines which other targets can depend on it. Targets can be - private (`//visibility:private`) Only targets in the same package can depend on it. - visible to specific packages (`//some/package:__pkg__`) Only targets within `//some/package` can depend on it. - visible to sub-packages (`//some/package:__subpackages__`) Only targets within or underneath `//some/package` can depend on it. - public (`//visibility:public`) Any target in any package can depend on it. Visibility should be kept as strict as possible to help maintain a clean dependency graph. Bazel files are written in a language called *Starlark*. It is very similar to Python. However, Starlark programs cannot perform arbitrary input and output, and build files are not allowed to use control structures (`for`, `if`, etc.), or define functions. These restrictions are in place to ensure hermeticity. [bazel_visibility]: https://docs.bazel.build/versions/master/be/common-definitions.html#common.visibility ## IntelliJ Integration Make sure to go through the Bazel setup section and to familiarize yourself with Bazel's core concepts as explained in the sections above, before you proceed to the IntelliJ integration. ### Setup If you use the IntelliJ IDE you should install the [Bazel integration plugin provided by Google][intellij_plugin]. Follow the [installation instructions][intellij_plugin_install] in the official documentation. In short: Install the plugin from within the IDE (`Settings > Plugins > Marketplace`, and search for 'Bazel'). Multiple Bazel plugins exist, make sure to select the Bazel plugin referencing [ij.bazel.build][intellij_plugin]. If the correct plugin does not exist in the list, then your IntelliJ version might be too recent, and the Bazel plugin might not have been upgraded to support it, yet. Check for version compatibility on the [Jetbrains plugin page][intellij_plugin_jetbrains]. At the time of writing the Bazel IntelliJ plugin version 2018.10.08.0.2 has been tested on IDEA Community 2018.2.5. [intellij_plugin]: https://ij.bazel.build/ [intellij_plugin_install]: https://ij.bazel.build/docs/bazel-plugin.html#getting-started [intellij_plugin_jetbrains]: https://plugins.jetbrains.com/plugin/8609-bazel ### Importing a project To import a Bazel project into IntelliJ select "Import Bazel Project" in the welcome dialog, or `File > Import Bazel Project` in the editor window. In the import dialog under "Workspace:" enter the path to the DAML repository root. The Bazel IntelliJ integration uses a *project view* file to define the list of directories and targets to make accessible in IntelliJ and to control other aspects of the project. Refer to the [official documentation][intellij_project_view] for a detailed description. [intellij_project_view]: https://ij.bazel.build/docs/project-views.html Here we will focus on two use-cases: "Import project view file" (recommended), and "Generate from BUILD file". #### Import project view file Check if the project you will be working on already has a project view file. These are typically stored under `project/path/.bazelproject`. If the project does not have a project view file yet, then you can generate one using the `bazel-project-view` tool in dev-env as follows: ``` bazel-project-view -o ledger/sandbox/.bazelproject ledger/sandbox ``` Choose the "Import Project View File" option and select the project view file that you would like to import. Then, click on "Next". The following dialog allows you to define the project name, or infer it, and to set the location of the project data directory. It also allows you to modify the imported project view file. This should not be necessary. If you do wish to modify the project view file, then refer to the next section for instructions. Click "Next" once you are ready. You will be able to modify the project view file later on as well. IntelliJ will now import the project. This process will take a while. #### Generate from BUILD file Choose the "Generate from BUILD file" option and select the `BUILD.bazel` file of the project that you will be working on. Then, click on "Next". The following dialog allows you to define the project name, or infer it, and to set the location of the project data directory. It also allows you to modify the default project view file. The default should have the following structure: ``` directories: path/to/project targets: //path/to/project/...:all additional_languages: # ... ``` Make sure to add Scala, or other languages that you require, to the `additional_languages` section. The section will be pre-populated with a list of comments specifying the automatically detected supported languages. Add any additional directories you would like accessible in the directory tree to the `directories` section. The default value in the `targets` section is a wild-card to select all project and sub-project targets. Note that wild-cards will not automatically generate *run configurations*. Run configurations are required to build targets or run tests from within IntelliJ. The details are explained below. If you wish to define a specific set of targets to work, then you can list these in the `targets` section. After these modifications the project view file might look like this: ``` directories: ledger/sandbox ... targets: //ledger/sandbox:sandbox ... additional_languages: scala ``` Click "Next" once you are ready. You will be able to modify the project view file later on as well. IntelliJ will now import the project. This process will take a while. ### Overview over Bazel IntelliJ Integration The IntelliJ interface should largely look the same as under SBT. However, the main menu will have an additional entry for Bazel, and a Bazel toolbar is provided for quick access to common tasks. The most commonly required operations are described below. Refer to the [plugin documentation][intellij_plugin_docs] for further information. [intellij_plugin_docs]: https://ij.bazel.build/docs/bazel-plugin.html #### Sync Project If you modified a project `BUILD.bazel` file, or the project view file, then click the "Sync Project with BUILD Files" button in the Bazel toolbar, or the `Sync > Sync Project with BUILD Files` entry in the Bazel menu to synchronize IntelliJ with those changes. #### Modify Project View Click `Project > Open Local Project View File` in the Bazel menu to open and modify the current project view file. You may need to sync the project for your changes to take effect. #### Build the Project Click `Build > Compile Project` in the Bazel menu to build the whole project. Click `Build > Compile "CURRENT FILE"` to compile only the current file. #### Run or Debug an Executable or Test Click on the drop-down menu in the Bazel tool bar and select the entry `Bazel run ` or `Bazel test `. If the executable or test you wish to run or debug is not in the list then follow instructions on adding a run configuration below first. Come back here when ready. The selected entry is a run configuration. Click the green arrow in the Bazel toolbar to run the executable or test. Click the green bug icon to debug the executable or test. #### Adding a Run Configuration If you wish to add an executable or test to the run configurations, then the simplest and most consistent way is to add the target to the project view file and sync the project. Otherwise, click on the drop-down menu in the Bazel tool bar and select "Edit Configurations...". Click the "+" in the upper left corner and select "Bazel Command" from the list. This will add a new entry to the "Bazel Command" sub-tree. Fill in the fields in the pane on the right side to define the run configuration: Give a suitable name, select the Bazel target, and choose the Bazel command (`run`, `test`, etc.). If applicable, you can also define Bazel command-line flags or command-line flags to the executable. Click on "Apply", or "OK" to add the run configuration. ### Known Issues #### Missing folders in project tree The "Project" pane contains a tree-view of the folders in the project. The "Compact Middle Packages" feature can cause intermediate folders to disappear, only showing their children in the tree-view. The workaround is to disable the feature by clicking on the gear icon in the "Project" pane and unchecking "Compact Middle Packages". Refer to the [issue tracker][project_tree_issue] for details. [project_tree_issue]: https://github.com/bazelbuild/intellij/issues/437 #### Rerun failed tests is broken IntelliJ allows to rerun only a single failed test-case by the click of a button. Unfortunately, this feature does not work with the Bazel plugin on Scala test-cases. Please refer to the [issue tracker][rerun_failed_tests_issue] for details. [rerun_failed_tests_issue]: https://github.com/bazelbuild/intellij/issues/446 ## Bazel Command Reference The following sections briefly list Bazel commands for the most common use-cases. Refer to the [official Bazel documentation][bazel_docs] for more detailed information. [bazel_docs]: https://docs.bazel.build/versions/master/bazel-overview.html ### Building Targets - Build all targets ``` bazel build //... ``` - Build an individual target ``` bazel build //ledger/sandbox:sandbox ``` ### Running Tests - Execute all tests ``` bazel test //... ``` - Execute a test suite ``` bazel test //ledger/sandbox:sandbox-scala-tests ``` - Show test output ``` bazel test //ledger/sandbox:sandbox-scala-tests --test_output=streamed ``` - Do not cache test results ``` bazel test //ledger/sandbox:sandbox-scala-tests --nocache_test_results ``` - Execute a specific Scala test-suite class ``` bazel test //ledger/sandbox:sandbox-scala-tests_test_suite_src_test_suite_scala_com_digitalasset_platform_sandbox_stores_ledger_sql_JdbcLedgerDaoSpec.scala ``` - Execute a test with a specific name ``` bazel test \ //ledger/sandbox:sandbox-scala-tests_test_suite_src_test_suite_scala_com_digitalasset_platform_sandbox_stores_ledger_sql_JdbcLedgerDaoSpec.scala \ --test_arg=-t \ --test_arg="JDBC Ledger DAO should be able to persist and load contracts without external offset" ``` - Pass an argument to a test case in a Scala test-suite ``` bazel test //ledger/sandbox:sandbox-scala-tests_test_suite_src_test_suite_scala_com_digitalasset_platform_sandbox_stores_ledger_sql_JdbcLedgerDaoSpec.scala \ --test_arg=-z \ --test_arg="should return true" ``` More broadly, for Scala tests you can pass through any of the args outlined in http://www.scalatest.org/user_guide/using_the_runner, separating into two instances of the --test-arg parameter as shown in the two examples above. ### Running Executables - Run an executable target ``` bazel run //ledger/sandbox:sandbox-binary ``` - Pass arguments to an executable target ``` bazel run //ledger/sandbox:sandbox-binary -- --help ``` ### Querying Targets The Bazel query language is described in detail in the [official Bazel documentation][bazel_query_lang]. This section will list a few common use-cases. Filters like `filter` or `kind` accept regular expressions. Query expressions can be combined using set operations like `intersect` or `union`. [bazel_query_lang]: https://docs.bazel.build/versions/master/query.html - List all targets underneath a directory ``` bazel query //ledger/... ``` - List all library targets underneath a directory ``` bazel query 'kind("library rule", //ledger/...)' ``` - List all Scala library targets underneath a directory ``` bazel query 'kind("scala.*library rule", //ledger/...)' ``` - List all test-suites underneath a directory ``` bazel query 'kind("test_suite", //ledger/...)' ``` - List all test-cases underneath a directory ``` bazel query 'tests(//ledger/...)' ``` - List all Java test-cases underneath a directory ``` bazel query 'kind("java", tests(//ledger/...))' ``` - List all Scala library dependencies of a target ``` bazel query 'kind("scala.*library rule", deps(//ledger/sandbox:sandbox))' ``` - Find available 3rd party dependencies ``` bazel query 'attr(visibility, public, filter(".*scalaz.*", //3rdparty/...))' ``` #### Dependency Graphs Bazel queries can also output dependency graphs between the targets that the query includes. These can then be rendered using Graphviz. - Graph all Scala library dependencies of a target ``` bazel query --noimplicit_deps 'kind(scala_library, deps(//ledger/sandbox:sandbox))' --output graph > graph.in dot -Tpng < graph.in > graph.png ``` The `--noimplicit_deps` flag excludes dependencies that are not explicitly listed in the `BUILD` file, but that are added by Bazel implicitly, e.g. the unused dependency checker added by `rules_scala`. ### Help - List available commands ``` bazel help ``` - Show help on a Bazel command ``` bazel help build ``` - Show details on each option ``` bazel help build --long ``` ## Continuous Build By continuous build we mean the ability to watch the repository for source file changes and rebuild or rerun targets when any relevant files change. The dev-env provides the tool `ibazel` for that purpose. Similar to Bazel it can be called with the commands `build`, `test`, or `run` on a specific target. It will perform the command and determine a list of relevant source files. Then, it will watch these files for changes and rerun the command on file change. For example: ``` ibazel test //ledger/sandbox:sandbox-scala-tests ``` Note, that this interacts well with Bazel's test result caching (which is activated by default). In the above example the outcome of tests whose sources didn't change will already be cached by Bazel and the tests won't be repeated. Refer to the [project README][bazel_watcher] for more information. [bazel_watcher]: https://github.com/bazelbuild/bazel-watcher#readme ## Haskell in Bazel In Bazel terminology, a directory containing a `BUILD.bazel` file is called a "package". Packages contain targets and `BUILD.bazel` files are where targets are defined. Mostly we are concerned in our `BUILD.bazel` files with writing rules to produce specific Haskell derived files (artifacts) from Haskell source file inputs. Of these, most are libraries, some are executables and some are tests. For Haskell, most `BUILD.bazel` files begin with a variation on the following: ``` load( "//bazel_tools:haskell.bzl", "da_haskell_library", "da_haskell_executable","da_haskell_test" ) ``` This directive loads from the `//bazel_tools` package, the rules [`da_haskell_library`](https://api.haskell.build/haskell/haskell.html#haskell_library) for building libraries, [`da_haskell_binary`](https://api.haskell.build/haskell/haskell.html#haskell_binary) for building executables and [`da_haskell_test`](https://api.haskell.build/haskell/haskell.html#haskell_test) for building test-suites. The `da_*` rules are DA specific overrides of the upstream `rules_haskell` rules. Their API docs can be found in `//bazel_tools/haskell.bzl`, or by executing the `bazel-api-docs` tool from `dev-env`. They mostly behave like the upstream rules, just adding some defaults, and adding a `hackage_deps` attribute (more on this below) for convenience. ### Library : `da_haskell_library` One specific library in the `daml-foundations` stack is `daml-ghc-compiler`. Here's a synopsis of its definition. ``` da_haskell_library( name = "daml-ghc-compiler", srcs = glob([ "src/**/*.hs", ]), src_strip_prefix = "src", deps = [ "//compiler/daml-lf-ast", "//compiler/daml-lf-proto", ... ], hackage_deps = [ "base", "bytestring", ... ], visibility = ["//visibility:public"], ) ``` To build this single target from the root of the DAML repository, the command would be: ``` bazel build //compiler/damlc/daml-compiler ``` since the `BUILD.bazel` that defines the target is in the `compiler/damlc` sub-folder of the root of the DA repository and the target `name` is `damlc`. Let's break this definition down: - `name`: A unique name for the target; - `srcs`: A list of Haskell source files; - `src_stip_prefix`: Directory in which the module hierarchy starts; - `deps`: A list of in-house Haskell or C library dependencies to be linked into the target; - `hackage_deps`: A list of external Haskell (Hackage) libraries to be linked into the target; - `visibility`: Define whether depending on this target by others is permissible. Note the use of the Bazel [`glob`](https://docs.bazel.build/versions/master/be/functions.html#glob) function to define the `srcs` argument allowing us to avoid having to enumerate all source files explicitly. The `**` part of the shown glob expression is Bazel syntax for any sub-path. Read more about `glob` [here](https://docs.bazel.build/versions/master/be/functions.html#glob). The `deps` argument in the above invocation can be interpreted as linking the libraries defined by the list of targets provided on the right hand side (when we say "package" here we mean Haskell package - i.e. library): - `//:ghc-lib` is the `ghc-lib` package defined in the root `BUILD` file, - `//compiler/daml-lf-ast` is the `daml-lf-ast` package defined in the `daml-foundations/daml-compiler/BUILD.bazel` file (that is, `//compiler/daml-lf-ast` is shorthand for `//compiler/daml-lf-ast:daml-lf-ast`) - Similarly, `//nix/third-party/proto3-suite` is the Haskell library `//nix/third-party/proto3-suite:proto3-suite` defined in the file `nix/third-party/proto3-suite/BUILD.bazel`. The `hackage_deps` argument details those Haskell packages (from Hackage) that the `daml-ghc-compiler` target depends upon. In this case that is `base`, `bytestring` and some other packages not shown. Finally, `visibility` is set to public so no errors will result should another target attempt to link `daml-ghc-compiler`. *[Note : Public visibility means that any other target from anywhere can depend on the target. To keep the dependency graph sane, its a good idea to keep visibility restrictive. See [here](https://docs.bazel.build/versions/master/be/common-definitions.html#common.visibility) for more detail.]* ### Executable : `da_haskell_binary` Here's the synopsis of the rule for the executable `daml-ghc`: ``` da_haskell_binary ( name = "daml-ghc", srcs = glob (["src/DA/Cli/**/*.hs", "src/DA/Test/**/*.hs"]) src_strip_prefix = "DA", main_function = "DA.Cli.GHC.Run.main", hackage_deps = [ "base", "time", ...], data = [ "//compiler/damlc/pkg-db" , ... ], deps = [ ":daml-ghc-compiler" , "//:ghc-lib" , "//compiler/daml-lf-ast" , ... ] , visibility = ["//visibility:public"] ) ``` Haskell binaries require a definition of the distinguished function `main`. The `main_function` argument allows us to express the qualified module path to the definition of `main` to use. The `data` argument in essence is a list of files needed by the target at runtime. Consult [this documentation](https://docs.bazel.build/versions/master/be/common-definitions.html#common.data) for more detail. The targets that are given on the right-hand-side are (long-hand) labels for "file-groups". Here's the one for `daml-stdlib-src` for example. ``` filegroup( name = "daml-stdlib-src", srcs = glob(["daml-stdlib/**"]), visibility = ["//visibility:public"] ) ``` Having looked at `deps` in the context of `haskell_library` there's not much more to say except note the `:daml-ghc-compiler` syntax for the depedency on `daml-ghc-compiler`. That is, targets defined in the same `BUILD.bazel` as the target being defined can be referred to by preceding their names with `:`. ### Test : `da_haskell_test` For an example of a test target, we turn to `//libs-haskell/da-hs-base:da-hs-base-tests`: ``` da_haskell_test( name = "da-hs-base-tests", src_strip_prefix = "src-tests", srcs = glob(["src-tests/**/*.hs"]), deps = [ ":da-hs-base", ], visibility = ["//visibility:public"], ) ``` There is nothing new in the above to expound upon here! How might you invoke that single target? Simple as this: ``` bazel test "//libs-haskell/da-hs-base:da-hs-base-tests" ``` More comprehensive documentation on the `bazel` command can be found [here](https://docs.bazel.build/versions/master/command-line-reference.html). ### Beyond defining targets If your work goes beyond simply adding targets to existing `BUILD.bazel` files and involves things like defining toolchains and external dependencies, then [this document](https://github.com/digital-asset/daml/blob/master/BAZEL-haskell.md) is for you! ## Documentation packages in Bazel 'daml-foundations' documentation resides in the DA git repository in sub-directories of the path `//daml-foundations/daml-tools/docs`. Each sub-directory there is a documentation "package". A documentation package contains a `BUILD.bazel` file. The rule for producing a documentation package under Bazel is `da_doc_package` and it is brought into the scope of a `BUILD.bazel` file with the following directive. ``` load ("//bazel_tools:docs.bzl", "da_doc_package") ``` ### Synopsis `da_doc_package (name, prepare, extra_sources)` Build a documentation package. Attributes: - `name` Required. A unique name for the package. - `prepare` Optional. If provided then it is interpreted as a bash script to be executed on the documentation sources before the bundle generation step (see below for what that means). - `extra_sources` Optional. Default value is the empty list. The output of `da_doc_package` with name `"foo"` is a bundle `sources.tar.gzip` in the path `//bazel-bin/daml-foundations/daml-tools/docs/foo`. The bundle for `"foo"` would be produced with the command: ``` bazel build //daml-foundations/daml-tools/docs/foo:foo ``` The contents of the bundle will be copies of files under the directory `//daml-foundations/daml-tools/doc/foo/sources`. ## Scala in Bazel In this section we will provide an overview of how Scala targets are defined in Bazel in this repository. This should provide enough information for most everyday development tasks. For more information refer to the Bazel porting guide for JVM developers (to be written as of now). For a general reference to `BUILD.bazel` file syntax refer to the [official documentation][build_file_docs]. Note, that `BUILD.bazel` and `BUILD` are both valid file names. However, `BUILD.bazel` is the preferred spelling. `BUILD` is the older spelling and still to be found in large parts of the documentation. [build_file_docs]: https://docs.bazel.build/versions/master/build-ref.html#BUILD_files Bazel targets are defined in `BUILD.bazel` files. For example `//ledger-client/ods:ods` is defined in `ledger-client/ods/BUILD.bazel`. First, we import the required rules and macros. For example, the following loads the `da_scala_library` macro defined in `//bazel_tools:scala.bzl`, and the `daml` rule defined in `ledger-client/daml.bzl`. The distinction of rules and macros is not important here. ``` load('//bazel_tools:scala.bzl', 'da_scala_library') load('//rules_daml:daml.bzl', 'daml') ``` ### Scala Libraries The macro `da_scala_library` is a convenience function that defines a Scala library and sets common compiler flags, plugins, etc. To explain it we will take an example instance and describe the individual attributes. For details refer to the [`rules_scala` project README][rules_scala_docs]. For a Java rules refer to the [official Bazel documentation][java_docs]. [rules_scala_docs]: https://github.com/bazelbuild/rules_scala#scala-rules-for-bazel [java_docs]: https://docs.bazel.build/versions/master/be/java.html#java-rules ``` da_scala_library( # Set the target name to 'ods'. name = 'ods', # Mark this target as public. # I.e. targets in other package can depend on it. visibility = ['//visibility:public'], # Define the target's source files by globbing. # The details of file globbing are explained here: # https://docs.bazel.build/versions/master/be/functions.html#glob srcs = glob(['src/main/**/*.scala'], exclude = [...]), # Define the target's resources by globbing. resources = glob(['src/main/resources/**/*']), # Define the target's dependencies. # These will appear in the compile-time classpath. # And the transient closure of `deps`, `runtime_deps`, and `exports` # will appear in the runtime classpath. deps = [ # A third party dependency. '//3rdparty/jvm/ch/qos/logback:logback_classic', # A dependency in the same workspace. '//ledger-client/nanobot-framework', # A dependency in the same package. ':ods-macro' ... ], # Define the target's runtime dependencies. # These will appear only in the runtime classpath. runtime_deps = [...], # List of exported targets. # E.g. if something depends on ':ods', it will also depend on ':ods-macro'. exports = [':ods-macro'], # Scalac compiler plugins to use for this target. # Note, that these have to be specified as JAR targets. # I.e. you cannot use `//3rdparty/jvm/org/scalameta/paradise_2_12_6` here. plugins = [ '//external:jar/org/scalameta/paradise_2_12_6', ], # Scalac compiler options to use for this target. scalacopts = ['-Xplugin-require:macroparadise'], # JVM flags to pass to the Scalac compiler. scalac_jvm_flags = ['-Xmx2G'], ) ``` ### Scala Macro Libraries If a Scala library defines macros that should be used by other Scala targets later on, then it has to be defined using `da_scala_macro_library`. Macros may not be defined and used within the same target. ### Scala Executables Scala executables are defined using `da_scala_binary`. It takes most of the same attributes that `da_scala_library` takes. Notable additional attributes are: - `main_class`: Name of the class defining the entry-point `main()`. - `jvm_flags`: Flags to pass to the JVM at runtime. - `data`: Files that are needed at runtime. In order to access such files at runtime you should use the utility library in `com.digitalasset.testing.BuildSystemSupport`. ### Scala Test Cases Scala test-suites are defined using `da_scala_test_suite`. It takes most of the same attributes as `da_scala_binary`. Note, that this macro will create one target for every single source file specified in `srcs`. The advantage is that separate test-cases can be addressed as separate Bazel targets and Bazel's test-result caching can be applied more beneficially. However, this means that test-suite source files may not depend on each other. A single Scala test-cases, potentially consisting of multiple source files, can be defined using `da_scala_test`. It is preferable to always use `da_scala_test_suite`, and define separate testing utility libraries using `da_scala_library` if test-cases depend on utility modules. ### Scala JMH Benchmarks Scala benchmarks based on the JMH toolkit can be defined using the `scala_benchmark_jmh` macro provided by `rules_scala`. It supports a restricted subset of the attributes of `da_scala_binary`, namely: `name`, `deps`, `srcs`, `scalacopts`, `resources` and `resource_jars`. The end result of building the benchmark is a Scala binary of the same name, which can be executed with `bazel run`. ### Java and Scala Deployment Bazel's builtin Java rules and `rules_scala` will automatically generate a fat JAR suitable for deployment for all your Java and Scala targets. For example, if you defined a Scala executable target called `foo`, then Bazel will generate the target `foo_deploy.jar` next to the regular `foo.jar` target. Building the `foo_deploy.jar` target will generate a self-contained fat JAR suitable to be passed to `java -jar`. ### DAML Packages DAML package targets are defined using the `daml` rule loaded from `//rules_daml:daml.bzl`. To explain it we will take an example instance and describe the individual attributes. ``` daml( name = "it-daml", # The main DAML file. This file will be passed to damlc. main_src = "src/it/resources/TestAll.daml", # Other DAML files that may be imported by the main DAML file. srcs = glob(["src/it/resources/**/*.daml"]), # The directory prefix under which to create the DAR tree. target_dir = "target/scala-2.12/resource_managed/it/dars", # The group ID. group = "com.digitalasset.sample", # The artifact ID. artifact = "test-all", # The package version. version = "0.1", # The package name. package = "com.digitalasset.sample", ) ``` This will compile and package the DAML code into a DAR file under the following target, where `` is the `group` attribute with `.` replaced by `/`. ``` :////-.dar, ``` For example: ``` :target/scala-2.12/resource_managed/it/dars/com/digitalasset/sample/test-all/0.1/test-all-0.1.dar ``` POM and SHA files will be stored in the same directory. Additionally, this will perform Scala code generation and bundle the generated Scala modules into a source JAR available under the following target. ``` .srcjar ``` For example: ``` it-daml.srcjar ``` ### DAML Executables The rule `daml_binary` is provided to generate executable targets that execute the DAML sandbox on a given DAR package. For example: ``` daml_binary( name = "ping-pong-exec", dar = ':target/repository/.../PingPong-0.1.dar', ) ``` Such a target can then be executed as follows, where arguments after `--` are passed to the DAML sandbox. ``` bazel run //ledger-client/nanobot-sample-app:ping-pong-exec -- --help ``` ### External Java and Scala Dependencies External dependencies are these that are not defined and built within the local workspace, but are defined in an external workspace in some way. The most common case are Maven JAR dependencies which are fetched from Artifactory. We distinguish direct and transitive dependencies. Direct dependencies are explicitly defined on targets in the local workspace. Most commonly on the `deps`, `runtime_deps`, `exports`, or `plugins` attributes. Transitive dependencies are introduced implicitly through direct dependencies, most commonly on another dependency's `exports` attribute. All direct Scala and Java dependencies are listed explicitly in the file `bazel-java-deps.bzl`. Each dependency is defined by its Maven coordinates. The `maven_install` repository rule calls Coursier to perform transitive dependency resolution and import the required artifacts into the Bazel build. The resolved versions are pinned in the file `maven_install.json`. Execute `bazel run @unpinned_maven//:pin` when you wish to update or add a new dependency. See [`rules_jvm_external`][rules_jvm_external] for details. [rules_jvm_external]: https://github.com/bazelbuild/rules_jvm_external#updating-maven_installjson ## Typescript in Bazel We are using [rules_typescript](https://github.com/bazelbuild/rules_typescript) to build typescript projects. It works in conjunction with rules_nodejs to provide access to npm packages. Please refer to the documentation in the above url for usage. For an example, please see `compiler/daml-extension/BUILD.bazel`. ## Protocol buffers in Bazel We use protocol buffers for DAML-LF and the Ledger API. The DAML-LF protocol buffer build rules can be found from //daml-lf/archive/BUILD.bazel. It produces bindings for Java and Haskell (via proto3-suite). Bazel provides built-in rules for protocol buffer bindings for Java and C++. See the following resources for more information on its usage: [Protocol Buffer Rules](https://docs.bazel.build/versions/master/be/protocol-buffer.html) [Blog post: Protocol Buffers in Bazel](https://blog.bazel.build/2017/02/27/protocol-buffers.html) The rules for haskell are currently ad-hoc genrules and use the proto3-suite's compile-proto-file program directly. Please refer to //daml-lf/archive/BUILD.bazel for example usage. If you find yourself writing similar rules, please take a moment to write some Starlark to abstract it out and document it here. Note that proto3-suite isn't compatible with protoc, so it is not currently possible to hook it up into the "proto_library" tooling. ## Known issues ### Unchanged Haskell library being rebuilt Unfortunately, [GHC builds are not deterministic](https://gitlab.haskell.org/ghc/ghc/issues/12262). This, coupled with the way Bazel works, may lead to Haskell libraries that have not been changed to be rebuilt. If the library sits at the base of the dependency graph, it may cause a ripple effect that forces you to rebuild most of the workspace without an actual need for it (`ghc-lib` is one example of this). **To work around this issue** you can clean the local and build cache, making sure you are fetching the GHC build artifacts from remote: bazel clean --expunge # clean the build cache rm -r .bazel-cache # clean the local cache This will also mean that changes made locally will need to be rebuilt, but it's likely that this will still result in a net positive gain on your build time. ### Working in environments with low or intermittent connectivity Bazel tries to leverage the remote cache to speed up the build process but this can turn out to work against you if you are working in an environment with low or intermittent connectivity. To disable fetching from the remote cache in such scenario, you can use the `--noremote_accept_cached` option.