24 KiB
Making a Release
Please read this document carefully.
At a high level, a release goes through the following steps. All releases must use the same version number for stable releases; there is more leeway for snapshots and RCs.
- A "daml repo" release, orchestrated from this repository.
- A "canton repo" release, where the code in the release commit must have a dependency on the artifacts produced by step 1.
- An "assembly repo" release, which combines the daml and canton release artifacts and creates the GitHub Release on the daml repo.
- A "daml-finance repo" release. This is independant of the other steps on a technical level.
- A "docs repo" release to publish documentation on docs.daml.com.
- Manual tests and approval of the final release artifacts.
Note that these are the technical steps to create the release artifacts. The release process as a whole is quite a bit larger and involves synchronization between various teams about exactly what we want to include in a release, when we want to release it, communication to various stakeholders, etc. For details on the broader process, see the release planning document.
For the more detailed technical steps, we need to distinguish between three cases:
a. A new minor release. b. A patch release on an existing minor version. c. The weekly snapshot.
The way in which we get to the final artifact differs based on the three cases above, but the manual testing steps are always the same. As such, they have their own section at the end of this document.
Minor Release
For a minor release, we will usually set a target date and a target scope. When
we get "close enough" to the target scope or date, we create the release
branches for the future release, branching off from main
. There is no hard rule about
when this step happens, and there may be a few days between repos.
In the daml repo, the release branch for minor version A.B
must be named
release/A.B.x
, e.g. release/2.7.x
. In the canton repo, the release
branch is named release-line-A.B
, e.g. release-line-2.7
. In both repos, we
have special "branch protection rules" set up in GitHub for branches with those
names.
Note: This also means you should not create branches with those (or similar) names if they are not meant to be release branches.
When the release branches contain all the required changes and there are no blockers left, we create a release candidate (RC). This step can be repeated multiple times: if issues are found with the current RC, we'll add patches to the release branches, and create a new RC.
Note: In most cases work should not be done on a release branch directly. Changes should go into the
main
branch first, and then be backported to the release branch.
After some time and testing (see the release planning document for details), the RC is accepted and we create a new release from the same code base.
RC version strings in the daml repo are indistinguishable from snapshot
names. In the canton repo, snapshots are usually names by their date of
production (e.g. 20230401
) whereas RCs contain an explicit rc
market (e.g.
2.6.0-rc2
). This can create some confusion, but is hard to change at this
time.
When you create the release branch in the daml repo, make a PR to
main
bumping [NIGHTLY_PREFIX
] accordingly.
The steps to create a release are:
-
In the daml repo, edit the
LATEST
file. The format of that file is a commit sha of the code you want to release (at this point typically the tip of the release branch), the version number you want to attribute to the build, and the wordsSPLIT_RELEASE
(for historical reason dating to pre-2.0 days). For a release candidate, one can generate a snapshot version string using therelease.sh
script. For example, to create a PR for a 2.0.0 release candidate:$ git fetch $ ./release.sh snapshot origin/release/2.0.x 2.0.0
You should put that line in the file in order to preserve semver ordering, and overwrite any existing snapshot with the same prefix.
Stable releases should use the same code as the last RC, so instead of adding a new line you should simply remove the
-snapshot.*
part of the version string on the appropriate line of theLATEST
file. -
Make a PR targeting the
main
branch with just that one line added, touching no other file. Add theStandard-Change
label to that PR. -
When the PR is merged, the build of the corresponding commit on
main
will create a "split release" bundle and push it to Artifactory. It should notify on#ci-failures-daml
on Slack. -
The next step is to request a build from the Canton team (on
#team-canton
) that relies on the RC snapshot. Once you have a canton release, you can proceed. -
Go to the assembly repo, and follow the instructions there to make a release using the Canton version that was just created. The
LATEST
file on the assembly repo only contains one version; it is safe to overwrite it, and to "go backwards" if needed. -
Once the
main
build of the assembly repo has finished, you should proceed with testing. You should open up this document in the branch of the release you're making, as testing instructions change over time. (Though rarely these days.)
Patch Release
On the technical side, a patch release is very similar to a minor release, except that:
- The non-technical steps of the broader process are much simpler. See the release planning document for details.
- Depending on the nature of the patch, we sometimes skip the RC process and go straight for a stable version number. This is a judgement call based on the specifics of the changes.
Patch releases are done from the corresponding release branch.
Weekly Snapshot
Note: You should not have to make any change to this repo.
The weekly snapshot relies on daily builds from both the canton and daml repos, so there is no need to create those manually.
-
On Tuesday, you should see a reminder on Slack (
#team-daml
) with the name of the release tester for the week. -
On Wednesday morning, a cron should create a release PR on the assembly repo, mentioning you. Please reach out to
@gary
on Slack if that's missing. -
Follow the instructions in the PR description. Merge the PR and wait for the corresponding
main
build to finish. -
Once the assembly build is finished, it should post a message to Slack with instructions on how to get the release artifacts.
-
Go to the Testing section of this file.
Note: Documentation for weekly snapshots should be published automatically by the docs repo cron. This is an hourly cron, however, and it can only run after the release artifacts have been created and pushed to GitHub releases on the daml repo, so it may take some time to appear.
Testing
This testing procedure starts once the release is listed on the releases page.
In the following notes, we assume that $VERSION
contains
the full version tag for the release you are testing - in other words, the full version as recorded on the Slack
#team-daml
message that is generated when closing the main
build PR.
For example, for the Slack message:
team_daml_notifs
Just published
2.4.0-snapshot.20220830.10494.0.4622de48
.For testing:
- Follow the instructions.
- Install on macOS/Linux with
curl -sSL https://get.daml.com/ | sh -s 2.4.0-snapshot.20220830.10494.0.4622de48
.- Install on Windows using this link.
we set $VERSION
to be 2.4.0-snapshot.20220830.10494.0.4622de48
.
-
On Windows, install the new SDK using the installer on the releases page. This will typically be the asset named
daml-sdk-$VERSION-windows.exe
(located on the DAML releases page). Please ensure that$VERSION
is expanded correctly!. -
On MacOS/Linux (please ensure that
$VERSION
is expanded correctly!):
curl -sSL https://get.daml.com/ | sh -s "$VERSION"
Tips for Windows testing in an ad-hoc machine
If you are part of the release rotation, you can create Windows VMs through the [ad-hoc] project. The created machine is a bit raw, though, so here are a few tips to help you along.
First we should clone the git repository https://github.com/DACH-NY/daml-language-ad-hoc and then enter the cloned repo.
If this is your first time doing this, edit
tf/main.tf
and add your username to themembers
field of thegoogle_project_iam_binding.machine_managers
resource. Generate and submit a PR with these changes. Once the PR has been accepted, you should now have permission to create GCP compute instances.Assuming
direnv
is installed, entering thedaml-language-ad-hoc
project directory will be sufficient to configure and install the extra software (e.g. the GCP SDK) required for your environment. Note that this could take a few minutes.A new GCP windows instance can be created by running
./ad-hoc.sh temp windows
inside thedaml-language-ad-hoc
project. This command prints IP address, username and password for the created Windows VM. Save this output. You will need this information later when you create an RDP connection.‼️ After starting, it's going to take some time for the machine to be configured (see notes below).
Before you may connect to this windows instance, you need to ensure that the VPN is connected. On Mac OSX you can do this by selecting the preconfigured Connect GCP Frankfurt full tunnel VPN profile.
If you're on a Mac, you can use Microsoft Remote Desktop to connect. This can be installed via the Mac App Store or directly here.
If you're on Linux, you can use Remmina.
Remmina notes: when creating an RDP connection, you may want to specify custom resolution. The default setting is to
use client resolution
. You may notice a failure due to color depth settings. You can adjust those in the settings panel right below the resolution settings.The ad-hoc machines take a bit of time to be available after being reported as created, so be patient for a bit if your first connection attempt(s) fail.
Once the Windows machine is up and running, use Firefox (in Windows) to download and install
daml-sdk-$VERSION-windows.exe
from https://github.com/digital-asset/daml/releases (please ensure$VERSION
is expanded correctly!.NOTE 1: Use Firefox for testing. Windows machines come with both Internet Explorer and Firefox installed. Do not make the mistake of trying to use Internet Explorer.
Ad-hoc machines also come with Node, VSCode and OpenJDK preinstalled, so you don't need to worry about those.
NOTE 2: After logging in, it takes some time for the machine to be configured. The script that installs Firefox, Node, VSCode and OpenJDK runs once the machine is available for login. The software you need should appear within about 10 minutes (an easy way to check is to try to open
D:\
, as this volume is created after all the software is installed).All the commands mentioned in this testing section can be run from a simple DOS prompt (start menu -> type "cmd" -> click "Command prompt").
At the end of your Windows testing session, please be sure to terminate the GCP instance by running
./ad-hoc.sh destroy $ID
. Here$ID
is the identity for your GCP instance - this is printed when you create your Windows instance.
-
Prerequisites for running the tests:
- Visual Studio Code, Java-SDK
- Node.js
- Just the bare install; no need to build C dependencies.
If you have
nix
installed, you can use a suitable version of nodejs by runningnix-shell -p nodejs-18_x
before running thenpm
commands below.
- Just the bare install; no need to build C dependencies.
If you have
- Maven
-
Run
daml version --assistant=yes
and verify that the new version is selected as the assistant version and the default version for new projects. -
Tests for the getting started guide (macOS/Linux and Windows). Note: if using a remote Windows VM and an RDP client that supports copy/paste, you can run through this on both Windows and your local unix in parallel fairly easily.
-
For these steps you will need the getting started documentation for the release that you are about to make. This documentation (for the release that you are testing) is published at
https://docs.daml.com/$VERSION/getting-started/index.html
. Please ensure that$VERSION
is expanded correctly before trying this link! -
daml new create-daml-app --template create-daml-app
-
cd create-daml-app
daml start
-
In a new terminal (with nodejs configured as above), from the
ui
folder:-
npm install
- if this command returns with an exit code of 0, errors may be safely ignored.
-
npm start
-
-
Open two browser windows (you want to see them simultaneously ideally) at
localhost:3000
. -
Log in as
alice
in the first window, log in asbob
in the second window. -
In the first window, where you are logged in as
Alice
, followBob
by typing their name in the dropdown (note that it will beBob
notbob
, the former is the global alias, the latter is the participant-local username). Verify thatBob
appears in the list of usersAlice
is following. Verify in the other browser window thatAlice
shows up inBob
’s network. -
In the second window, where you are logged in as
Bob
, followAlice
by selecting it in the dropdown. Verify thatAlice
appears in the list of usersBob
is following. Verify in the other browser window thatBob
shows up inAlice
’s network. -
Open the "Your first feature" section of the Getting Started Guide, e.g., from
https://docs.daml.com/$VERSION/getting-started/first-feature.html
if you did not build docs locally. -
In a third terminal window, run
daml studio --replace=always
from the project root directory. This should open Visual Studio Code. In the editor, opendaml/User.daml
. -
Copy the
Message
template from the documentation to the end ofUser.daml
. -
Copy the
SendMessage
choice from the documentation to theUser
template below theFollow
choice. -
Save your changes and close VSCode.
-
In the first terminal window (where
daml start
is running), press 'r' (respectively 'r' + 'Enter' on Windows). -
In the third terminal window, run
daml studio
. -
Create
MessageList.tsx
,MessageEdit.tsx
and modifyMainView.tsx
as described in the documentation. -
Verify that you do not see errors in the typescript code in VSCode.
-
Save your changes and close VSCode.
-
As before, open two browser windows at
localhost:3000
and log in asalice
andbob
. -
Make
Alice
followBob
. -
From
Bob
, select Alice in theSelect a follower
dropdown, inserthi alice
in the message field and click onSend
. -
Verify that
Alice
has received the message in the other window. -
Make
Bob
followAlice
. -
From
Alice
, select Bob in theSelect a follower
dropdown, inserthi bob
in the message field and click onSend
. -
Verify that
Bob
has received the message in the other window. -
You can now close both browser windows and both running processes (
daml start
andnpm start
). -
Don't forget to run this on the other platform! E.g. if you just ran through on Linux or macOS, you still need to run on Windows, and vice versa. For testing on Windows instances, please refer to the Tips for Windows testing in an ad-hoc machine notes above.
-
-
Run through the following test plan on Windows. This is slightly shortened to make testing faster and since most issues are not platform specific.
- Run
daml new myproject
to create a new project and switch to it usingcd myproject
. - Run
daml start
. - Open your browser at
http://localhost:7500
, verify that you can log in as alice and there is one contract, and that the template list containsMain:Asset
among other templates. - Kill
daml start
withCtrl-C
. - Run
daml studio --replace=always
and opendaml/Main.daml
. Verify that the script result appears within 30 seconds.- you will need to click on the Script results link in the open VS code window in order to verify this
- Add
+
at the end of line 26 after(PartyIdHint "Alice")
and verify that you get an error on line 27.
- Run
-
On your PR (the one that triggered the release process: on daml for 1.x releases, and on assembly for 2.x releases), add the comment:
Manual tests passed on Windows.
-
Tests for
quickstart-java
(Linux/macOS)While this is no longer the default in the Getting Started Guide, we still test it since the process covers things not covered by the new Getting Started Guide (e.g. Navigator, Scripts, Maven artifacts, etc.)
-
Create a new project with
daml new quickstart --template quickstart-java
and switch to it usingcd quickstart
. -
Verify the new version is specified in
daml.yaml
as thesdk-version
. -
Run
daml start
. Your browser should be opened automatically athttp://localhost:7500
.-
Login as
alice
and verify that there is 1 contract. -
In the templates section, verify that the templates list contains
Iou:Iou
,Iou:IouTransfer
, andIouTrade:IouTrade
among other templates.
-
-
Close the tab and kill
daml start
usingCtrl-C
. -
Run
daml build
. -
In 3 separate terminals (each being in the
quickstart-java
project directory), run:-
In Terminal 1 run
daml sandbox --port 6865
-
In Terminal 2, run each of the following:
-
daml ledger upload-dar --host localhost --port 6865 .daml/dist/quickstart-0.0.1.dar
daml script --ledger-host localhost --ledger-port 6865 --dar .daml/dist/quickstart-0.0.1.dar --script-name Main:initialize --output-file output.json
-
cat output.json
and verify that the output looks like this:["Alice::NAMESPACE", "EUR_Bank::NAMESPACE"]
where
NAMESPACE
is some randomly generated series of hex digits. -
daml navigator server localhost 6865 --port 7500
-
-
In Terminal 3, run:
daml codegen java && mvn compile exec:java@run-quickstart -Dparty=$(cat output.json | sed 's/\[\"//' | sed 's/".*//')
Note that this step scrapes the
Alice::NAMESPACE
party name from theoutput.json
produced in the previous steps.Note: It takes some time (typically around half-an-hour) for our artifacts to be available on Maven Central. If you try running the last command before the artifacts are available, you will get a "not found" error. Trying to build again in the next 24 hours will result in:
Failure to find ... was cached in the local repository, resolution will not be reattempted until the update interval of digitalasset-releases has elapsed or updates are forced
This is Maven telling you it has locally cached that "not found" result and will consider it valid for 24h. To bypass that and force Maven to try the network call again, add a
-U
option, as inmvn compile exec:java@run-quickstart -U
. Note that this is required to bypass your local cache of the failure; it will not be required for a user trying to run the quickstart after the artifacts have been published.Another common problem is that artifacts fail to resolve because of custom Maven settings. Check your
~/.m2/settings.xml
configuration and try disabling them temporarily.
-
-
Point your browser to
http://localhost:7500
, login asalice
and verify that there is 1 contract, 1 owned IOU, and the templates list containsIou:Iou
,Iou:IouTransfer
, andIouTrade:IouTrade
among other templates. -
Check that
curl http://localhost:8080/iou
returns:{"0":{"issuer":"EUR_Bank::NAMESPACE","owner":"Alice::NAMESPACE","currency":"EUR","amount":100.0000000000,"observers":[]}}
where NAMESPACE is again the series of hex digits that you saw before.
-
Kill all processes.
-
Run
daml studio --replace=always
. This should open VSCode and trigger the Daml extension that's bundled with the new SDK version. (The new VSCode extension will not be in the marketplace at this point.) -
Open
daml/Main.daml
. -
Click on
Script results
aboveinitialize
(in the code) and wait for the script results to appear. -
Add
+
at the end of line 14, after(PartyIdHint "Alice")
and confirm you get an error in line 15. -
Add
1
after the+
and confirm you get a type error in line 14, which says thatScript Party
does not matchInt
. -
Delete the
+1
and thee
in the second"Alice"
and verify that the script results are updated to the misspelled name. -
Right click on
eurBank
in line 28 and verify that "Go to Definition" takes you to the definition in line 17. -
Close all files.
Note: when running
daml studio --replace=always
, you force the installation of the VSCode extension bundled with the Daml SDK, and disable the auto-upgrade mechanism in VSCode. To instruct VSCode to go back to the published version of the extension, including auto-upgrades, you can rundaml studio --replace=published
. -
-
On your PR (the one that triggered the release process: on daml for 1.x releases, and on assembly for 2.x releases), add the comment:
Manual tests passed on [Linux/macOS].
specifying which platform you tested on.
-
If the release is bad, ask Gary to delete the release from the releases page. Mention why it is bad as a comment on your PR, and stop the process here.
Note that the Standard-Change label must remain on the PR, even if the release has failed.
-
Announce the release on
#product-daml
on Slack. For a stable release, direct people to the release blog post. If there were any errors during testing, but we decided to keep the release anyway, report those on the PR and include a link to the PR in the announcement.
For a stable release, you need to additionally:
-
Go to the releases page and remove the prerelease marker on the release. Also change the text to
See [the release notes blog]() for details.
adding in the direct link to this version's release notes. Documentation for this release will be added to docs.daml.com on the next hour. -
Coordinate with product (& marketing) for the relevant public announcements (Daml Forum, Twitter, etc.).
-
Documentation is published automatically once the release is public on GitHub, though this runs on an hourly cron.
Thanks for making a release!