mirror of
https://github.com/digital-asset/daml.git
synced 2024-09-20 09:17:43 +03:00
5a117dc358
Context ======= After multiple discussions about our current release schedule and process, we've come to the conclusion that we need to be able to make a distinction between technical snapshots and marketing releases. In other words, we need to be able to create a bundle for early adopters to test without making it an officially-supported version, and without necessarily implying everyone should go through the trouble of upgrading. The underlying goal is to have less frequent but more stable "official" releases. This PR is a proposal for a new release process designed under the following constraints: - Reuse as much as possible of the existing infrastructure, to minimize effort but also chances of disruptions. - Have the ability to create "snapshot"/"nightly"/... releases that are not meant for general public consumption, but can still be used by savvy users without jumping through too many extra hoops (ideally just swapping in a slightly-weirder version string). - Have the ability to promote an existing snapshot release to "official" release status, with as few changes as possible in-between, so we can be confident that the official release is what we tested as a prerelease. - Have as much of the release pipeline shared between the two types of releases, to avoid discovering non-transient problems while trying to promote a snapshot to an official release. - Triggerring a release should still be done through a PR, so we can keep the same approval process for SOC2 auditability. The gist of this proposal is to replace the current `VERSION` file with a `LATEST` file, which would have the following format: ``` ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7 ``` This file would be maintained with a script to reduce manual labor in producing the version string. Other than that, the process will be largely the same, with releases triggered by changes to this `LATEST` and the release notes files. Version numbers =============== Because one of the goals is to reduce the velocity of our published version numbers, we need a different version scheme for our snapshot releases. Fortunately, most version schemes have some support for that; unfortunately, the SDK sits at the intersection of three different version schemes that have made incompatible choices. Without going into too much detail: - Semantic versioning (which we chose as the version format for the SDK version number) allows for "prerelease" version numbers as well as "metadata"; an example of a complete version string would be `1.2.3-nightly.201+server12.43`. The "main" part of the version string always has to have 3 numbers separated by dots; the "prerelease" (after the `-` but before the `+`) and the "metadata" (after the `+`) parts are optional and, if present, must consist of one or more segments separated by dots, where a segment can be either a number or an alphanumeric string. In terms of ordering, metadata is irrelevant and any version with a prerelease string is before the corresponding "main" version string alone. Amongst prereleases, segments are compared in order with purely numeric ones compared as numbers and mixed ones compared lexicographically. So 1.2.3 is more recent than 1.2.3-1, which is itself less recent than 1.2.3-2. - Maven version strings are any number of segments separated by a `.`, a `-`, or a transition between a number and a letter. Version strings are compared element-wise, with numeric segments being compared as numbers. Alphabetic segments are treated specially if they happen to be one of a handful of magic words (such as "alpha", "beta" or "snapshot" for example) which count as "qualifiers"; a version string with a qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`, which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a special ordering amongst qualifiers. Other alphabetic segments are compared alphabetically and count as being "after" their prefix (`1.2.3-really-final-this-time` counts as being released after `1.2.3`). - GHC package numbers are comprised of any number of numeric segments separated by `.`, plus an optional (though deprecated) alphanumeric "version tag" separated by a `-`. I could not find any official documentation on ordering for the version tag; numeric segments are compared as numbers. - npm uses semantic versioning so that is covered already. After much more investigation than I'd care to admit, I have come up with the following compromise as the least-bad solution. First, obviously, the version string for stable/marketing versions is going to be "standard" semver, i.e. major.minor.patch, all numbers, which works, and sorts as expected, for all three schemes. For snapshot releases, we shall use the following (semver) format: ``` 0.13.53-alpha.20200214.3025.ef5d32b7 ``` where the components are, respectively: - `0.13.53`: the expected version string of the next "stable" release. - `alpha`: a marker that hopefully scares people enough. - `20200214`: the date of the release commit, which _MUST_ be on master. - `3025`: the number of commits in master up to the release commit (included). Because we have a linear, append-only master branch, this uniquely identifies the commit. - `ef5d32b7ù : the first 8 characters of the release commit sha. This is not strictly speaking necessary, but makes it a lot more convenient to identify the commit. The main downsides of this format are: 1. It is not a valid format for GHC packages. We do not publish GHC packages from the SDK (so far we have instead opted to release our Haskell code as separate packages entirely), so this should not be an issue. However, our SDK version currently leaks to `ghc-pkg` as the version string for the stdlib (and prim) packages. This PR addresses that by tweaking the compiler to remove the offending bits, so `ghc-pkg` would see the above version number as `0.13.53.20200214.3025`, which should be enough to uniquely identify it. Note that, as far as I could find out, this number would never be exposed to users. 2. It is rather long, which I think is good from a human perspective as it makes it more scary. However, I have been told that this may be long enough to cause issues on Windows by pushing us past the max path size limitation of that "OS". I suggest we try it and see what happens. The upsides are: - It clearly indicates it is an unstable release (`alpha`). - It clearly indicates how old it is, by including the date. - To humans, it is immediately obvious which version is "later" even if they have the same date, allowing us to release same-day patches if needed. (Note: that is, commits that were made on the same day; the release date itself is irrelevant here.) - It contains the git sha so the commit built for that release is immediately obvious. - It sorts correctly under all schemes (modulo the modification for GHC). Alternatives I considered: - Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would be accepted by all schemes, but will not sort as expected under semantic versioning (though Maven will be fine). I have no idea how it will sort under GHC. - Not having any non-numeric component, e.g. `0.13.53.20200214.3025`. This is not valid semantic versioning and is therefore rejected by npm. - Not having detailed info: just go with `0.13.53-snapshot`. This is what is generally done in the Java world, but we then lose track of what version is actually in use and I'm concerned about bug reports. This would also not let us publish to the main Maven repo (at least not more than once), as artifacts there are supposed to be immutable. - No having a qualifier: `0.13.53-3025` would be acceptable to all three version formats. However, it would not clearly indicate to humans that it is not meant as a stable version, and would sort differently under semantic versioning (which counts it as a prerelease, i.e. before `0.13.53`) than under maven (which counts it as a patch, so after `0.13.53`). - Just counting releases: `0.13.53-alpha.1`, where we just count the number of prereleases in-between `0.13.52` and the next. This is currently the fallback plan if Windows path length causes issues. It would be less convenient to map releases to commits, but it could still be done via querying the history of the `LATEST` file. Release notes ============= > Note: We have decided not to have release notes for snapshot releases. Release notes are a bit tricky. Because we want the ability to make snapshot releases, then later on promote them to stable releases, it follows that we want to build commits from the past. However, if we decide post-hoc that a commit is actually a good candidate for a release, there is no way that commit can have the appropriate release notes: it cannot know what version number it's getting, and, moreover, we now track changes in commit messages. And I do not think anyone wants to go back to the release notes file being a merge bottleneck. But release notes need to be published to the releases blog upon releasing a stable version, and the docs website needs to be updated and include them. The only sensible solution here is to pick up the release notes as of the commit that triggers the release. As the docs cron runs asynchronously, this means walking down the git history to find the relevant commit. > Note: We could probably do away with the asynchronicity at this point. > It was originally included to cover for the possibility of a release > failing. If we are releasing commits from the past after they have been > tested, this should not be an issue anymore. If the docs generation were > part of the synchronous release step, it would have direct access to the > correct release notes without having to walk down the git history. > > However, I think it is more prudent to keep this change as a future step, > after we're confident the new release scheme does indeed produce much more > reliable "stable" releases. New release process =================== Just like releases are currently controlled mostly by detecting changes to the `VERSION` file, the new process will be controlled by detecting changes to the `LATEST` file. The format of that file will include both the version string and the corresponding SHA. Upon detecting a change to the `LATEST` file, CI will run the entire release process, just like it does now with the VERSION file. The main differences are: 1. Before running the release step, CI will checkout the commit specified in the LATEST file. This requires separating the release step from the build step, which in my opinion is cleaner anyway. 2. The `//:VERSION` Bazel target is replaced by a repository rule that gets the version to build from an environment variable, with a default of `0.0.0` to remain consistent with the current `daml-head` behaviour. Some of the manual steps will need to be skipped for a snapshot release. See amended `release/RELEASE.md` in this commit for details. The main caveat of this approach is that the official release will be a different binary from the corresponding snapshot. It will have been built from the same source, but with a different version string. This is somewhat mitigated by Bazel caching, meaning any build step that does not depend on the version string should use the cache and produce identical results. I do not think this can be avoided when our artifact includes its own version number. I must note, though, that while going through the changes required after removing the `VERSION` file, I have been quite surprised at the sheer number of things that actually depend on the SDK version number. I believe we should look into reducing that over time. CHANGELOG_BEGIN CHANGELOG_END
1044 lines
37 KiB
Markdown
1044 lines
37 KiB
Markdown
# Bazel User Guide
|
|
|
|
This document explains how to use the Bazel build system on the DAML repository
|
|
from a users perspective. I.e. assuming the project you are working on has
|
|
already been ported to Bazel.
|
|
|
|
This guide does not cover how to port a new project to Bazel. Please refer to
|
|
the [Bazel JVM Porting Guide][bazel_jvm_guide] if you intend to port a JVM
|
|
project to Bazel.
|
|
|
|
[bazel_jvm_guide]: ./BAZEL-JVM.md
|
|
|
|
## Setup
|
|
|
|
This section goes through the required steps for a basic but fully functioning
|
|
setup of the Bazel build system for work on the DAML repository. Additional setup
|
|
as for the IntelliJ integration is listed in its own section below.
|
|
|
|
### Bazel Executable
|
|
|
|
Bazel is incorporated in the dev-env. If the [dev-env is setup
|
|
correctly][dev_env_guide] and `dev-env/bin` is in your `$PATH`, then Bazel
|
|
should be ready to use.
|
|
|
|
[dev_env_guide]: ./README.md
|
|
|
|
## Build and Test
|
|
|
|
Once setup is complete, you can build the whole repository with the following
|
|
command.
|
|
|
|
```
|
|
bazel build //...
|
|
```
|
|
|
|
You can run all hermetic tests in the repository with the following command.
|
|
|
|
```
|
|
bazel test //...
|
|
```
|
|
|
|
## Bazel Core Concepts
|
|
|
|
If you are unfamiliar with Bazel it is recommended that you read the official
|
|
[Concepts and Terminology guide][bazel_core_concepts]. Here we will only
|
|
provide a brief overview which may serve as a refresher.
|
|
|
|
[bazel_core_concepts]: https://docs.bazel.build/versions/master/build-ref.html
|
|
|
|
In short, the `daml` repository is a Bazel *workspace*. It contains a `WORKSPACE`
|
|
file, which defines external dependencies. The workspace contains several
|
|
*packages*. A package is a directory that contains a `BUILD.bazel` or `BUILD`
|
|
file. Each package holds multiple *targets*. Targets are either *files* under
|
|
the package directory or *rules* defined in the `BUILD.bazel` file. You can
|
|
address a target by a *label* of the form `//path/to/package:target`. For
|
|
example, `//ledger/sandbox:sandbox`. Here `sandbox` is a target in the package
|
|
`ledger/sandbox`. It is defined in the file `ledger/sandbox/BUILD.bazel`
|
|
using `da_scala_library` as shown below.
|
|
|
|
```
|
|
da_scala_library(
|
|
name = "sandbox",
|
|
srcs = glob(["src/main/scala/**/*.scala"]),
|
|
resources =
|
|
glob(
|
|
["src/main/resources/**/*"],
|
|
# Do not include logback.xml into the library: let the user
|
|
# of the sandbox-as-a-library decide how to log.
|
|
exclude = ["src/main/resources/logback.xml"],
|
|
) + [
|
|
"//:MVN_VERSION",
|
|
],
|
|
tags = ["maven_coordinates=com.digitalasset.platform:sandbox:__VERSION__"],
|
|
visibility = [
|
|
"//visibility:public",
|
|
],
|
|
runtime_deps = [
|
|
"@maven//:ch_qos_logback_logback_classic",
|
|
"@maven//:ch_qos_logback_logback_core",
|
|
],
|
|
deps = compileDependencies,
|
|
)
|
|
```
|
|
|
|
The arguments to `da_scala_library` are called *attributes*. These define the
|
|
name of the target, the sources it is compiled from, its dependencies, etc.
|
|
Note, that Bazel build rules are hermetic. I.e. only explicitly declared
|
|
dependencies will be available during execution. In particular, if a rule
|
|
depends on additional data files, then these have to be declared dependencies
|
|
as well. For example using the `resources` or the `data` attributes. The
|
|
details depend on the rule in question.
|
|
|
|
The following rules are commonly used in this repository. For Scala projects
|
|
`da_scala_library`, `da_scala_test_suite`, `da_scala_binary`. For Java projects
|
|
`java_library`, `java_test_suite`, `java_binary`. For DAML projects `daml`.
|
|
|
|
Labels can point to a specific target, or to a set of targets using a
|
|
wild-card. The following wild-card patterns are recognized.
|
|
|
|
- Ellipsis (`//some/package/...`):
|
|
All rule targets within or underneath `some/package`.
|
|
- All (`//some/package:all`):
|
|
All rule targets within `some/package`.
|
|
- Star (`//some/package:*`):
|
|
All rule or file targets within `some/package`.
|
|
|
|
So far we have talked about targets defined in the current workspace. Bazel
|
|
also has a notion of external workspaces. Targets in external workspaces are
|
|
labelled as `@workspace_name//path/to/package:target`.
|
|
|
|
Targets have a *visibility* attribute that determines which other targets can
|
|
depend on it. Targets can be
|
|
|
|
- private (`//visibility:private`)
|
|
Only targets in the same package can depend on it.
|
|
- visible to specific packages (`//some/package:__pkg__`)
|
|
Only targets within `//some/package` can depend on it.
|
|
- visible to sub-packages (`//some/package:__subpackages__`)
|
|
Only targets within or underneath `//some/package` can depend on it.
|
|
- public (`//visibility:public`)
|
|
Any target in any package can depend on it.
|
|
|
|
Visibility should be kept as strict as possible to help maintain a clean
|
|
dependency graph.
|
|
|
|
Bazel files are written in a language called *Starlark*. It is very similar to
|
|
Python. However, Starlark programs cannot perform arbitrary input and output,
|
|
and build files are not allowed to use control structures (`for`, `if`, etc.),
|
|
or define functions. These restrictions are in place to ensure hermeticity.
|
|
|
|
[bazel_visibility]: https://docs.bazel.build/versions/master/be/common-definitions.html#common.visibility
|
|
|
|
## IntelliJ Integration
|
|
|
|
Make sure to go through the Bazel setup section and to familiarize yourself
|
|
with Bazel's core concepts as explained in the sections above, before you
|
|
proceed to the IntelliJ integration.
|
|
|
|
### Setup
|
|
|
|
If you use the IntelliJ IDE you should install the [Bazel integration plugin
|
|
provided by Google][intellij_plugin]. Follow the [installation
|
|
instructions][intellij_plugin_install] in the official documentation. In short:
|
|
Install the plugin from within the IDE (`Settings > Plugins > Marketplace`, and
|
|
search for 'Bazel'). Multiple Bazel plugins exist, make sure to select the Bazel
|
|
plugin referencing [ij.bazel.build][intellij_plugin].
|
|
|
|
If the correct plugin does not exist in the list, then your IntelliJ version
|
|
might be too recent, and the Bazel plugin might not have been upgraded to
|
|
support it, yet. Check for version compatibility on the [Jetbrains plugin
|
|
page][intellij_plugin_jetbrains]. At the time of writing the Bazel IntelliJ
|
|
plugin version 2018.10.08.0.2 has been tested on IDEA Community 2018.2.5.
|
|
|
|
[intellij_plugin]: https://ij.bazel.build/
|
|
[intellij_plugin_install]: https://ij.bazel.build/docs/bazel-plugin.html#getting-started
|
|
[intellij_plugin_jetbrains]: https://plugins.jetbrains.com/plugin/8609-bazel
|
|
|
|
### Importing a project
|
|
|
|
To import a Bazel project into IntelliJ select "Import Bazel Project" in the
|
|
welcome dialog, or `File > Import Bazel Project` in the editor window. In the
|
|
import dialog under "Workspace:" enter the path to the DAML repository root.
|
|
|
|
The Bazel IntelliJ integration uses a *project view* file to define the list of
|
|
directories and targets to make accessible in IntelliJ and to control other
|
|
aspects of the project. Refer to the [official
|
|
documentation][intellij_project_view] for a detailed description.
|
|
|
|
[intellij_project_view]: https://ij.bazel.build/docs/project-views.html
|
|
|
|
Here we will focus on two use-cases: "Import project view file" (recommended),
|
|
and "Generate from BUILD file".
|
|
|
|
#### Import project view file
|
|
|
|
Check if the project you will be working on already has a project view file.
|
|
These are typically stored under `project/path/.bazelproject`. If the project
|
|
does not have a project view file yet, then you can generate one using the
|
|
`bazel-project-view` tool in dev-env as follows:
|
|
|
|
```
|
|
bazel-project-view -o ledger/sandbox/.bazelproject ledger/sandbox
|
|
```
|
|
|
|
Choose the "Import Project View File" option and select the project view file
|
|
that you would like to import. Then, click on "Next".
|
|
|
|
The following dialog allows you to define the project name, or infer it, and
|
|
to set the location of the project data directory. It also allows you to modify
|
|
the imported project view file. This should not be necessary. If you do wish to
|
|
modify the project view file, then refer to the next section for instructions.
|
|
|
|
Click "Next" once you are ready. You will be able to modify the project view
|
|
file later on as well. IntelliJ will now import the project. This process will
|
|
take a while.
|
|
|
|
#### Generate from BUILD file
|
|
|
|
Choose the "Generate from BUILD file" option and select the `BUILD.bazel` file
|
|
of the project that you will be working on. Then, click on "Next".
|
|
|
|
The following dialog allows you to define the project name, or infer it, and to
|
|
set the location of the project data directory. It also allows you to modify
|
|
the default project view file. The default should have the following structure:
|
|
|
|
```
|
|
directories:
|
|
path/to/project
|
|
|
|
targets:
|
|
//path/to/project/...:all
|
|
|
|
additional_languages:
|
|
# ...
|
|
```
|
|
|
|
Make sure to add Scala, or other languages that you require, to the
|
|
`additional_languages` section. The section will be pre-populated with a list
|
|
of comments specifying the automatically detected supported languages.
|
|
|
|
Add any additional directories you would like accessible in the directory tree
|
|
to the `directories` section.
|
|
|
|
The default value in the `targets` section is a wild-card to select all project
|
|
and sub-project targets. Note that wild-cards will not automatically generate
|
|
*run configurations*. Run configurations are required to build targets or run
|
|
tests from within IntelliJ. The details are explained below.
|
|
|
|
If you wish to define a specific set of targets to work, then you can list
|
|
these in the `targets` section.
|
|
|
|
After these modifications the project view file might look like this:
|
|
|
|
```
|
|
directories:
|
|
ledger/sandbox
|
|
...
|
|
|
|
targets:
|
|
//ledger/sandbox:sandbox
|
|
...
|
|
|
|
additional_languages:
|
|
scala
|
|
```
|
|
|
|
Click "Next" once you are ready. You will be able to modify the project view
|
|
file later on as well. IntelliJ will now import the project. This process will
|
|
take a while.
|
|
|
|
### Overview over Bazel IntelliJ Integration
|
|
|
|
The IntelliJ interface should largely look the same as under SBT. However, the
|
|
main menu will have an additional entry for Bazel, and a Bazel toolbar is
|
|
provided for quick access to common tasks.
|
|
|
|
The most commonly required operations are described below. Refer to the [plugin
|
|
documentation][intellij_plugin_docs] for further information.
|
|
|
|
[intellij_plugin_docs]: https://ij.bazel.build/docs/bazel-plugin.html
|
|
|
|
#### Sync Project
|
|
|
|
If you modified a project `BUILD.bazel` file, or the project view file, then
|
|
click the "Sync Project with BUILD Files" button in the Bazel toolbar, or the
|
|
`Sync > Sync Project with BUILD Files` entry in the Bazel menu to synchronize
|
|
IntelliJ with those changes.
|
|
|
|
#### Modify Project View
|
|
|
|
Click `Project > Open Local Project View File` in the Bazel menu to open and
|
|
modify the current project view file. You may need to sync the project for your
|
|
changes to take effect.
|
|
|
|
#### Build the Project
|
|
|
|
Click `Build > Compile Project` in the Bazel menu to build the whole project.
|
|
Click `Build > Compile "CURRENT FILE"` to compile only the current file.
|
|
|
|
#### Run or Debug an Executable or Test
|
|
|
|
Click on the drop-down menu in the Bazel tool bar and select the entry `Bazel
|
|
run <your target>` or `Bazel test <your target>`. If the executable or test you
|
|
wish to run or debug is not in the list then follow instructions on adding a
|
|
run configuration below first. Come back here when ready.
|
|
|
|
The selected entry is a run configuration. Click the green arrow in the Bazel
|
|
toolbar to run the executable or test. Click the green bug icon to debug the
|
|
executable or test.
|
|
|
|
#### Adding a Run Configuration
|
|
|
|
If you wish to add an executable or test to the run configurations, then the
|
|
simplest and most consistent way is to add the target to the project view file
|
|
and sync the project.
|
|
|
|
Otherwise, click on the drop-down menu in the Bazel tool bar and select "Edit
|
|
Configurations...". Click the "+" in the upper left corner and select "Bazel
|
|
Command" from the list. This will add a new entry to the "Bazel Command"
|
|
sub-tree. Fill in the fields in the pane on the right side to define the run
|
|
configuration: Give a suitable name, select the Bazel target, and choose the
|
|
Bazel command (`run`, `test`, etc.). If applicable, you can also define Bazel
|
|
command-line flags or command-line flags to the executable. Click on "Apply",
|
|
or "OK" to add the run configuration.
|
|
|
|
### Known Issues
|
|
|
|
#### Missing folders in project tree
|
|
|
|
The "Project" pane contains a tree-view of the folders in the project. The
|
|
"Compact Middle Packages" feature can cause intermediate folders to disappear,
|
|
only showing their children in the tree-view. The workaround is to disable the
|
|
feature by clicking on the gear icon in the "Project" pane and unchecking
|
|
"Compact Middle Packages". Refer to the [issue tracker][project_tree_issue] for
|
|
details.
|
|
|
|
[project_tree_issue]: https://github.com/bazelbuild/intellij/issues/437
|
|
|
|
#### Rerun failed tests is broken
|
|
|
|
IntelliJ allows to rerun only a single failed test-case by the click of a
|
|
button. Unfortunately, this feature does not work with the Bazel plugin on
|
|
Scala test-cases. Please refer to the [issue tracker][rerun_failed_tests_issue]
|
|
for details.
|
|
|
|
[rerun_failed_tests_issue]: https://github.com/bazelbuild/intellij/issues/446
|
|
|
|
## Bazel Command Reference
|
|
|
|
The following sections briefly list Bazel commands for the most common
|
|
use-cases. Refer to the [official Bazel documentation][bazel_docs] for more
|
|
detailed information.
|
|
|
|
[bazel_docs]: https://docs.bazel.build/versions/master/bazel-overview.html
|
|
|
|
### Building Targets
|
|
|
|
- Build all targets
|
|
|
|
```
|
|
bazel build //...
|
|
```
|
|
|
|
- Build an individual target
|
|
|
|
```
|
|
bazel build //ledger/sandbox:sandbox
|
|
```
|
|
|
|
### Running Tests
|
|
|
|
- Execute all tests
|
|
|
|
```
|
|
bazel test //...
|
|
```
|
|
|
|
- Execute a test suite
|
|
|
|
```
|
|
bazel test //ledger/sandbox:sandbox-scala-tests
|
|
```
|
|
|
|
- Show test output
|
|
|
|
```
|
|
bazel test //ledger/sandbox:sandbox-scala-tests --test_output=streamed
|
|
```
|
|
|
|
- Do not cache test results
|
|
|
|
```
|
|
bazel test //ledger/sandbox:sandbox-scala-tests --nocache_test_results
|
|
```
|
|
|
|
- Execute a specific Scala test-suite class
|
|
|
|
```
|
|
bazel test //ledger/sandbox:sandbox-scala-tests_test_suite_src_test_suite_scala_com_digitalasset_platform_sandbox_stores_ledger_sql_JdbcLedgerDaoSpec.scala
|
|
```
|
|
|
|
- Execute a test with a specific name
|
|
|
|
```
|
|
bazel test \
|
|
//ledger/sandbox:sandbox-scala-tests_test_suite_src_test_suite_scala_com_digitalasset_platform_sandbox_stores_ledger_sql_JdbcLedgerDaoSpec.scala \
|
|
--test_arg=-t \
|
|
--test_arg="JDBC Ledger DAO should be able to persist and load contracts without external offset"
|
|
```
|
|
|
|
- Pass an argument to a test case in a Scala test-suite
|
|
|
|
```
|
|
bazel test //ledger/sandbox:sandbox-scala-tests_test_suite_src_test_suite_scala_com_digitalasset_platform_sandbox_stores_ledger_sql_JdbcLedgerDaoSpec.scala \
|
|
--test_arg=-z \
|
|
--test_arg="should return true"
|
|
```
|
|
|
|
More broadly, for Scala tests you can pass through any of the args outlined in http://www.scalatest.org/user_guide/using_the_runner, separating into two instances of the --test-arg parameter as shown in the two examples above.
|
|
|
|
### Running Executables
|
|
|
|
- Run an executable target
|
|
|
|
```
|
|
bazel run //ledger/sandbox:sandbox-binary
|
|
```
|
|
|
|
- Pass arguments to an executable target
|
|
|
|
```
|
|
bazel run //ledger/sandbox:sandbox-binary -- --help
|
|
```
|
|
|
|
### Querying Targets
|
|
|
|
The Bazel query language is described in detail in the [official Bazel
|
|
documentation][bazel_query_lang]. This section will list a few common
|
|
use-cases. Filters like `filter` or `kind` accept regular expressions. Query
|
|
expressions can be combined using set operations like `intersect` or `union`.
|
|
|
|
[bazel_query_lang]: https://docs.bazel.build/versions/master/query.html
|
|
|
|
- List all targets underneath a directory
|
|
|
|
```
|
|
bazel query //ledger/...
|
|
```
|
|
|
|
- List all library targets underneath a directory
|
|
|
|
```
|
|
bazel query 'kind("library rule", //ledger/...)'
|
|
```
|
|
|
|
- List all Scala library targets underneath a directory
|
|
|
|
```
|
|
bazel query 'kind("scala.*library rule", //ledger/...)'
|
|
```
|
|
|
|
- List all test-suites underneath a directory
|
|
|
|
```
|
|
bazel query 'kind("test_suite", //ledger/...)'
|
|
```
|
|
|
|
- List all test-cases underneath a directory
|
|
|
|
```
|
|
bazel query 'tests(//ledger/...)'
|
|
```
|
|
|
|
- List all Java test-cases underneath a directory
|
|
|
|
```
|
|
bazel query 'kind("java", tests(//ledger/...))'
|
|
```
|
|
|
|
- List all Scala library dependencies of a target
|
|
|
|
```
|
|
bazel query 'kind("scala.*library rule", deps(//ledger/sandbox:sandbox))'
|
|
```
|
|
|
|
- Find available 3rd party dependencies
|
|
|
|
```
|
|
bazel query 'attr(visibility, public, filter(".*scalaz.*", //3rdparty/...))'
|
|
```
|
|
|
|
#### Dependency Graphs
|
|
|
|
Bazel queries can also output dependency graphs between the targets that the
|
|
query includes. These can then be rendered using Graphviz.
|
|
|
|
- Graph all Scala library dependencies of a target
|
|
|
|
```
|
|
bazel query --noimplicit_deps 'kind(scala_library, deps(//ledger/sandbox:sandbox))' --output graph > graph.in
|
|
dot -Tpng < graph.in > graph.png
|
|
```
|
|
|
|
The `--noimplicit_deps` flag excludes dependencies that are not explicitly
|
|
listed in the `BUILD` file, but that are added by Bazel implicitly, e.g.
|
|
the unused dependency checker added by `rules_scala`.
|
|
|
|
### Help
|
|
|
|
- List available commands
|
|
|
|
```
|
|
bazel help
|
|
```
|
|
|
|
- Show help on a Bazel command
|
|
|
|
```
|
|
bazel help build
|
|
```
|
|
|
|
- Show details on each option
|
|
|
|
```
|
|
bazel help build --long
|
|
```
|
|
|
|
## Continuous Build
|
|
|
|
By continuous build we mean the ability to watch the repository for source file
|
|
changes and rebuild or rerun targets when any relevant files change. The
|
|
dev-env provides the tool `ibazel` for that purpose. Similar to Bazel it can be
|
|
called with the commands `build`, `test`, or `run` on a specific target. It
|
|
will perform the command and determine a list of relevant source files. Then,
|
|
it will watch these files for changes and rerun the command on file change. For
|
|
example:
|
|
|
|
```
|
|
ibazel test //ledger/sandbox:sandbox-scala-tests
|
|
```
|
|
|
|
Note, that this interacts well with Bazel's test result caching (which is
|
|
activated by default). In the above example the outcome of tests whose sources
|
|
didn't change will already be cached by Bazel and the tests won't be repeated.
|
|
|
|
Refer to the [project README][bazel_watcher] for more information.
|
|
|
|
[bazel_watcher]: https://github.com/bazelbuild/bazel-watcher#readme
|
|
|
|
## Haskell in Bazel
|
|
|
|
In Bazel terminology, a directory containing a `BUILD.bazel` file is
|
|
called a "package". Packages contain targets and `BUILD.bazel` files
|
|
are where targets are defined. Mostly we are concerned in our
|
|
`BUILD.bazel` files with writing rules to produce specific Haskell
|
|
derived files (artifacts) from Haskell source file inputs. Of these,
|
|
most are libraries, some are executables and some are tests.
|
|
|
|
For Haskell, most `BUILD.bazel` files begin with a variation on the
|
|
following:
|
|
```
|
|
load( "//bazel_tools:haskell.bzl",
|
|
"da_haskell_library", "da_haskell_executable","da_haskell_test" )
|
|
```
|
|
This directive loads from the `//bazel_tools` package, the rules
|
|
[`da_haskell_library`](https://api.haskell.build/haskell/haskell.html#haskell_library)
|
|
for building libraries,
|
|
[`da_haskell_binary`](https://api.haskell.build/haskell/haskell.html#haskell_binary)
|
|
for building executables and
|
|
[`da_haskell_test`](https://api.haskell.build/haskell/haskell.html#haskell_test)
|
|
for building test-suites. The `da_*` rules are DA specific overrides
|
|
of the upstream `rules_haskell` rules. Their API docs can be found in
|
|
`//bazel_tools/haskell.bzl`, or by executing the `bazel-api-docs` tool
|
|
from `dev-env`. They mostly behave like the upstream rules, just
|
|
adding some defaults, and adding a `hackage_deps` attribute (more on
|
|
this below) for convenience.
|
|
|
|
### Library : `da_haskell_library`
|
|
|
|
One specific library in the `daml-foundations` stack is
|
|
`daml-ghc-compiler`. Here's a synopsis of its definition.
|
|
```
|
|
da_haskell_library(
|
|
name = "daml-ghc-compiler",
|
|
srcs = glob([
|
|
"src/**/*.hs",
|
|
]),
|
|
src_strip_prefix = "src",
|
|
deps = [
|
|
"//compiler/daml-lf-ast",
|
|
"//compiler/daml-lf-proto",
|
|
...
|
|
],
|
|
hackage_deps = [
|
|
"base",
|
|
"bytestring",
|
|
...
|
|
],
|
|
visibility = ["//visibility:public"],
|
|
)
|
|
```
|
|
To build this single target from the root of the DAML repository, the
|
|
command would be:
|
|
```
|
|
bazel build //compiler/damlc/daml-compiler
|
|
```
|
|
since the `BUILD.bazel` that defines the target is in the
|
|
`compiler/damlc` sub-folder of the root of the DA
|
|
repository and the target `name` is `damlc`.
|
|
|
|
Let's break this definition down:
|
|
- `name`:
|
|
A unique name for the target;
|
|
- `srcs`:
|
|
A list of Haskell source files;
|
|
- `src_stip_prefix`:
|
|
Directory in which the module hierarchy starts;
|
|
- `deps`:
|
|
A list of in-house Haskell or C library dependencies to be linked
|
|
into the target;
|
|
- `hackage_deps`:
|
|
A list of external Haskell (Hackage) libraries to be linked into
|
|
the target;
|
|
- `visibility`:
|
|
Define whether depending on this target by others is permissible.
|
|
|
|
Note the use of the Bazel
|
|
[`glob`](https://docs.bazel.build/versions/master/be/functions.html#glob)
|
|
function to define the `srcs` argument allowing us to avoid having to
|
|
enumerate all source files explicitly. The `**` part of the shown glob
|
|
expression is Bazel syntax for any sub-path. Read more about `glob`
|
|
[here](https://docs.bazel.build/versions/master/be/functions.html#glob).
|
|
|
|
The `deps` argument in the above invocation can be interpreted as
|
|
linking the libraries defined by the list of targets provided on the
|
|
right hand side (when we say "package" here we mean Haskell package -
|
|
i.e. library):
|
|
- `//:ghc-lib` is the `ghc-lib` package defined in the
|
|
root `BUILD` file,
|
|
- `//compiler/daml-lf-ast` is the
|
|
`daml-lf-ast` package defined in the
|
|
`daml-foundations/daml-compiler/BUILD.bazel` file (that is,
|
|
`//compiler/daml-lf-ast` is shorthand
|
|
for
|
|
`//compiler/daml-lf-ast:daml-lf-ast`)
|
|
- Similarly, `//nix/third-party/proto3-suite` is the Haskell library
|
|
`//nix/third-party/proto3-suite:proto3-suite` defined in the file
|
|
`nix/third-party/proto3-suite/BUILD.bazel`.
|
|
|
|
The `hackage_deps` argument details those Haskell packages (from
|
|
Hackage) that the `daml-ghc-compiler` target depends upon. In this case
|
|
that is `base`, `bytestring` and some other packages not
|
|
shown. Finally, `visibility` is set to public so no errors will result
|
|
should another target attempt to link `daml-ghc-compiler`. *[Note : Public
|
|
visibility means that any other target from anywhere can depend on the
|
|
target. To keep the dependency graph sane, its a good idea to keep
|
|
visibility restrictive. See
|
|
[here](https://docs.bazel.build/versions/master/be/common-definitions.html#common.visibility)
|
|
for more detail.]*
|
|
|
|
### Executable : `da_haskell_binary`
|
|
|
|
Here's the synopsis of the rule for the executable `daml-ghc`:
|
|
```
|
|
da_haskell_binary (
|
|
name = "daml-ghc",
|
|
srcs = glob (["src/DA/Cli/**/*.hs", "src/DA/Test/**/*.hs"])
|
|
src_strip_prefix = "DA",
|
|
main_function = "DA.Cli.GHC.Run.main",
|
|
hackage_deps = [ "base", "time", ...],
|
|
data = [
|
|
"//compiler/damlc/pkg-db"
|
|
, ...
|
|
],
|
|
deps = [
|
|
":daml-ghc-compiler"
|
|
, "//:ghc-lib"
|
|
, "//compiler/daml-lf-ast"
|
|
, ...
|
|
]
|
|
, visibility = ["//visibility:public"]
|
|
)
|
|
```
|
|
Haskell binaries require a definition of the distinguished function
|
|
`main`. The `main_function` argument allows us to express the
|
|
qualified module path to the definition of `main` to use.
|
|
|
|
The `data` argument in essence is a list of files needed by the target
|
|
at runtime. Consult [this
|
|
documentation](https://docs.bazel.build/versions/master/be/common-definitions.html#common.data)
|
|
for more detail. The targets that are given on the right-hand-side
|
|
are (long-hand) labels for "file-groups". Here's the one for
|
|
`daml-stdlib-src` for example.
|
|
```
|
|
filegroup(
|
|
name = "daml-stdlib-src",
|
|
srcs = glob(["daml-stdlib/**"]),
|
|
visibility = ["//visibility:public"]
|
|
)
|
|
```
|
|
|
|
Having looked at `deps` in the context of `haskell_library` there's
|
|
not much more to say except note the `:daml-ghc-compiler` syntax for the
|
|
depedency on `daml-ghc-compiler`. That is, targets defined in the same
|
|
`BUILD.bazel` as the target being defined can be referred to by
|
|
preceding their names with `:`.
|
|
|
|
### Test : `da_haskell_test`
|
|
|
|
For an example of a test target, we turn to
|
|
`//libs-haskell/da-hs-base:da-hs-base-tests`:
|
|
```
|
|
da_haskell_test(
|
|
name = "da-hs-base-tests",
|
|
src_strip_prefix = "src-tests",
|
|
srcs = glob(["src-tests/**/*.hs"]),
|
|
deps = [
|
|
":da-hs-base",
|
|
],
|
|
visibility = ["//visibility:public"],
|
|
)
|
|
```
|
|
There is nothing new in the above to expound upon here! How might you
|
|
invoke that single target? Simple as this:
|
|
```
|
|
bazel test "//libs-haskell/da-hs-base:da-hs-base-tests"
|
|
```
|
|
More comprehensive documentation on the `bazel` command can be found
|
|
[here](https://docs.bazel.build/versions/master/command-line-reference.html).
|
|
|
|
### Beyond defining targets
|
|
|
|
If your work goes beyond simply adding targets to existing
|
|
`BUILD.bazel` files and involves things like defining toolchains and
|
|
external dependencies, then [this
|
|
document](https://github.com/digital-asset/daml/blob/master/BAZEL-haskell.md)
|
|
is for you!
|
|
|
|
## Documentation packages in Bazel
|
|
|
|
'daml-foundations' documentation resides in the DA git repository in
|
|
sub-directories of the path `//daml-foundations/daml-tools/docs`. Each
|
|
sub-directory there is a documentation "package". A documentation
|
|
package contains a `BUILD.bazel` file.
|
|
|
|
The rule for producing a documentation package under Bazel is
|
|
`da_doc_package` and it is brought into the scope of a `BUILD.bazel`
|
|
file with the following directive.
|
|
```
|
|
load ("//bazel_tools:docs.bzl", "da_doc_package")
|
|
```
|
|
|
|
### Synopsis
|
|
|
|
`da_doc_package (name, prepare, extra_sources)`
|
|
|
|
Build a documentation package.
|
|
|
|
Attributes:
|
|
- `name`
|
|
Required. A unique name for the package.
|
|
- `prepare`
|
|
Optional. If provided then it is interpreted as a bash script to be
|
|
executed on the documentation sources before the bundle generation
|
|
step (see below for what that means).
|
|
- `extra_sources`
|
|
Optional. Default value is the empty list.
|
|
|
|
The output of `da_doc_package` with name `"foo"` is a bundle
|
|
`sources.tar.gzip` in the path
|
|
`//bazel-bin/daml-foundations/daml-tools/docs/foo`. The bundle for `"foo"` would be produced with the command:
|
|
```
|
|
bazel build //daml-foundations/daml-tools/docs/foo:foo
|
|
```
|
|
The contents of the bundle will be copies of files under the directory
|
|
`//daml-foundations/daml-tools/doc/foo/sources`.
|
|
|
|
## Scala in Bazel
|
|
|
|
In this section we will provide an overview of how Scala targets are defined in
|
|
Bazel in this repository. This should provide enough information for most
|
|
everyday development tasks. For more information refer to the Bazel porting
|
|
guide for JVM developers (to be written as of now). For a general reference to
|
|
`BUILD.bazel` file syntax refer to the [official
|
|
documentation][build_file_docs]. Note, that `BUILD.bazel` and `BUILD` are both
|
|
valid file names. However, `BUILD.bazel` is the preferred spelling. `BUILD` is
|
|
the older spelling and still to be found in large parts of the documentation.
|
|
|
|
[build_file_docs]: https://docs.bazel.build/versions/master/build-ref.html#BUILD_files
|
|
|
|
Bazel targets are defined in `BUILD.bazel` files. For example
|
|
`//ledger-client/ods:ods` is defined in `ledger-client/ods/BUILD.bazel`. First,
|
|
we import the required rules and macros. For example, the following loads the
|
|
`da_scala_library` macro defined in `//bazel_tools:scala.bzl`, and the `daml`
|
|
rule defined in `ledger-client/daml.bzl`. The distinction of rules and macros
|
|
is not important here.
|
|
|
|
```
|
|
load('//bazel_tools:scala.bzl', 'da_scala_library')
|
|
load('//rules_daml:daml.bzl', 'daml')
|
|
```
|
|
|
|
### Scala Libraries
|
|
|
|
The macro `da_scala_library` is a convenience function that defines a Scala
|
|
library and sets common compiler flags, plugins, etc. To explain it we will
|
|
take an example instance and describe the individual attributes. For details
|
|
refer to the [`rules_scala` project README][rules_scala_docs]. For a Java rules
|
|
refer to the [official Bazel documentation][java_docs].
|
|
|
|
[rules_scala_docs]: https://github.com/bazelbuild/rules_scala#scala-rules-for-bazel
|
|
[java_docs]: https://docs.bazel.build/versions/master/be/java.html#java-rules
|
|
|
|
```
|
|
da_scala_library(
|
|
|
|
# Set the target name to 'ods'.
|
|
name = 'ods',
|
|
|
|
# Mark this target as public.
|
|
# I.e. targets in other package can depend on it.
|
|
visibility = ['//visibility:public'],
|
|
|
|
# Define the target's source files by globbing.
|
|
# The details of file globbing are explained here:
|
|
# https://docs.bazel.build/versions/master/be/functions.html#glob
|
|
srcs = glob(['src/main/**/*.scala'], exclude = [...]),
|
|
|
|
# Define the target's resources by globbing.
|
|
resources = glob(['src/main/resources/**/*']),
|
|
|
|
# Define the target's dependencies.
|
|
# These will appear in the compile-time classpath.
|
|
# And the transient closure of `deps`, `runtime_deps`, and `exports`
|
|
# will appear in the runtime classpath.
|
|
deps = [
|
|
# A third party dependency.
|
|
'//3rdparty/jvm/ch/qos/logback:logback_classic',
|
|
# A dependency in the same workspace.
|
|
'//ledger-client/nanobot-framework',
|
|
# A dependency in the same package.
|
|
':ods-macro'
|
|
...
|
|
],
|
|
|
|
# Define the target's runtime dependencies.
|
|
# These will appear only in the runtime classpath.
|
|
runtime_deps = [...],
|
|
|
|
# List of exported targets.
|
|
# E.g. if something depends on ':ods', it will also depend on ':ods-macro'.
|
|
exports = [':ods-macro'],
|
|
|
|
# Scalac compiler plugins to use for this target.
|
|
# Note, that these have to be specified as JAR targets.
|
|
# I.e. you cannot use `//3rdparty/jvm/org/scalameta/paradise_2_12_6` here.
|
|
plugins = [
|
|
'//external:jar/org/scalameta/paradise_2_12_6',
|
|
],
|
|
|
|
# Scalac compiler options to use for this target.
|
|
scalacopts = ['-Xplugin-require:macroparadise'],
|
|
|
|
# JVM flags to pass to the Scalac compiler.
|
|
scalac_jvm_flags = ['-Xmx2G'],
|
|
)
|
|
```
|
|
|
|
### Scala Macro Libraries
|
|
|
|
If a Scala library defines macros that should be used by other Scala targets
|
|
later on, then it has to be defined using `da_scala_macro_library`. Macros may
|
|
not be defined and used within the same target.
|
|
|
|
### Scala Executables
|
|
|
|
Scala executables are defined using `da_scala_binary`. It takes most of the
|
|
same attributes that `da_scala_library` takes. Notable additional attributes
|
|
are:
|
|
|
|
- `main_class`:
|
|
Name of the class defining the entry-point `main()`.
|
|
- `jvm_flags`:
|
|
Flags to pass to the JVM at runtime.
|
|
- `data`:
|
|
Files that are needed at runtime. In order to access such files at runtime
|
|
you should use the utility library in
|
|
`com.digitalasset.testing.BuildSystemSupport`.
|
|
|
|
### Scala Test Cases
|
|
|
|
Scala test-suites are defined using `da_scala_test_suite`. It takes most of the
|
|
same attributes as `da_scala_binary`.
|
|
|
|
Note, that this macro will create one target for every single source file
|
|
specified in `srcs`. The advantage is that separate test-cases can be addressed
|
|
as separate Bazel targets and Bazel's test-result caching can be applied more
|
|
beneficially. However, this means that test-suite source files may not depend
|
|
on each other.
|
|
|
|
A single Scala test-cases, potentially consisting of multiple source files, can
|
|
be defined using `da_scala_test`. It is preferable to always use
|
|
`da_scala_test_suite`, and define separate testing utility libraries using
|
|
`da_scala_library` if test-cases depend on utility modules.
|
|
|
|
### Scala JMH Benchmarks
|
|
|
|
Scala benchmarks based on the JMH toolkit can be defined using the
|
|
`scala_benchmark_jmh` macro provided by `rules_scala`. It supports a restricted
|
|
subset of the attributes of `da_scala_binary`, namely: `name`, `deps`, `srcs`,
|
|
`scalacopts`, `resources` and `resource_jars`.
|
|
|
|
The end result of building the benchmark is a Scala binary of the same name,
|
|
which can be executed with `bazel run`.
|
|
|
|
### Java and Scala Deployment
|
|
|
|
Bazel's builtin Java rules and `rules_scala` will automatically generate a fat
|
|
JAR suitable for deployment for all your Java and Scala targets. For example,
|
|
if you defined a Scala executable target called `foo`, then Bazel will generate
|
|
the target `foo_deploy.jar` next to the regular `foo.jar` target. Building the
|
|
`foo_deploy.jar` target will generate a self-contained fat JAR suitable to be
|
|
passed to `java -jar`.
|
|
|
|
### DAML Packages
|
|
|
|
DAML package targets are defined using the `daml` rule loaded from
|
|
`//rules_daml:daml.bzl`. To explain it we will take an example instance and
|
|
describe the individual attributes.
|
|
|
|
```
|
|
daml(
|
|
name = "it-daml",
|
|
# The main DAML file. This file will be passed to damlc.
|
|
main_src = "src/it/resources/TestAll.daml",
|
|
# Other DAML files that may be imported by the main DAML file.
|
|
srcs = glob(["src/it/resources/**/*.daml"]),
|
|
# The directory prefix under which to create the DAR tree.
|
|
target_dir = "target/scala-2.12/resource_managed/it/dars",
|
|
# The group ID.
|
|
group = "com.digitalasset.sample",
|
|
# The artifact ID.
|
|
artifact = "test-all",
|
|
# The package version.
|
|
version = "0.1",
|
|
# The package name.
|
|
package = "com.digitalasset.sample",
|
|
)
|
|
```
|
|
|
|
This will compile and package the DAML code into a DAR file under the following
|
|
target, where `<group-dir>` is the `group` attribute with `.` replaced by `/`.
|
|
|
|
```
|
|
:<target_dir>/<group-dir>/<artifact>/<version>/<artifact>-<version>.dar,
|
|
```
|
|
|
|
For example:
|
|
|
|
```
|
|
:target/scala-2.12/resource_managed/it/dars/com/digitalasset/sample/test-all/0.1/test-all-0.1.dar
|
|
```
|
|
|
|
POM and SHA files will be stored in the same directory.
|
|
|
|
Additionally, this will perform Scala code generation and bundle the generated
|
|
Scala modules into a source JAR available under the following target.
|
|
|
|
```
|
|
<name>.srcjar
|
|
```
|
|
|
|
For example:
|
|
|
|
```
|
|
it-daml.srcjar
|
|
```
|
|
|
|
### DAML Executables
|
|
|
|
The rule `daml_binary` is provided to generate executable targets that execute
|
|
the DAML sandbox on a given DAR package. For example:
|
|
|
|
```
|
|
daml_binary(
|
|
name = "ping-pong-exec",
|
|
dar = ':target/repository/.../PingPong-0.1.dar',
|
|
)
|
|
```
|
|
|
|
Such a target can then be executed as follows, where arguments after `--` are
|
|
passed to the DAML sandbox.
|
|
|
|
```
|
|
bazel run //ledger-client/nanobot-sample-app:ping-pong-exec -- --help
|
|
```
|
|
|
|
### External Java and Scala Dependencies
|
|
|
|
External dependencies are these that are not defined and built within the local
|
|
workspace, but are defined in an external workspace in some way. The most
|
|
common case are Maven JAR dependencies which are fetched from Artifactory.
|
|
|
|
We distinguish direct and transitive dependencies. Direct dependencies are
|
|
explicitly defined on targets in the local workspace. Most commonly on the
|
|
`deps`, `runtime_deps`, `exports`, or `plugins` attributes. Transitive
|
|
dependencies are introduced implicitly through direct dependencies, most
|
|
commonly on another dependency's `exports` attribute.
|
|
|
|
All direct Scala and Java dependencies are listed explicitly in the file
|
|
`bazel-java-deps.bzl`. Each dependency is defined by its Maven coordinates. The
|
|
`maven_install` repository rule calls Coursier to perform transitive dependency
|
|
resolution and import the required artifacts into the Bazel build.
|
|
|
|
The resolved versions are pinned in the file `maven_install.json`. Execute
|
|
`bazel run @unpinned_maven//:pin` when you wish to update or add a new
|
|
dependency. See [`rules_jvm_external`][rules_jvm_external] for details.
|
|
|
|
[rules_jvm_external]: https://github.com/bazelbuild/rules_jvm_external#updating-maven_installjson
|
|
|
|
## Typescript in Bazel
|
|
|
|
We are using [rules_typescript](https://github.com/bazelbuild/rules_typescript) to build
|
|
typescript projects. It works in conjunction with rules_nodejs to provide access to
|
|
npm packages.
|
|
|
|
Please refer to the documentation in the above url for usage.
|
|
For an example, please see `compiler/daml-extension/BUILD.bazel`.
|
|
|
|
## Protocol buffers in Bazel
|
|
|
|
We use protocol buffers for DAML-LF and the Ledger API. The DAML-LF protocol
|
|
buffer build rules can be found from //daml-lf/archive/BUILD.bazel.
|
|
It produces bindings for Java and Haskell (via proto3-suite).
|
|
|
|
Bazel provides built-in rules for protocol buffer bindings for Java and C++.
|
|
See the following resources for more information on its usage:
|
|
[Protocol Buffer Rules](https://docs.bazel.build/versions/master/be/protocol-buffer.html)
|
|
[Blog post: Protocol Buffers in Bazel](https://blog.bazel.build/2017/02/27/protocol-buffers.html)
|
|
|
|
The rules for haskell are currently ad-hoc genrules and use the proto3-suite's compile-proto-file
|
|
program directly. Please refer to //daml-lf/archive/BUILD.bazel for example usage.
|
|
If you find yourself writing similar rules, please take a moment to write some Starlark to abstract
|
|
it out and document it here. Note that proto3-suite isn't compatible with protoc, so it is not currently
|
|
possible to hook it up into the "proto_library" tooling.
|
|
|
|
## Known issues
|
|
|
|
### Unchanged Haskell library being rebuilt
|
|
|
|
Unfortunately, [GHC builds are not deterministic](https://gitlab.haskell.org/ghc/ghc/issues/12262). This, coupled with the way Bazel works, may lead to Haskell libraries that have not been changed to be rebuilt. If the library sits at the base of the dependency graph, it may cause a ripple effect that forces you to rebuild most of the workspace without an actual need for it (`ghc-lib` is one example of this).
|
|
|
|
**To work around this issue** you can clean the local and build cache, making sure you are fetching the GHC build artifacts from remote:
|
|
|
|
bazel clean --expunge # clean the build cache
|
|
rm -r .bazel-cache # clean the local cache
|
|
|
|
This will also mean that changes made locally will need to be rebuilt, but it's likely that this will still result in a net positive gain on your build time.
|
|
|
|
### Working in environments with low or intermittent connectivity
|
|
|
|
Bazel tries to leverage the remote cache to speed up the build process but this can turn out to work against you if you are working in an environment with low or intermittent connectivity. To disable fetching from the remote cache in such scenario, you can use the `--noremote_accept_cached` option.
|
|
|