This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@robin-da is in charge of this release.
Commit log:
```
90c5ce703a Optimize mapOptional and add more efficient findOptional (#9214)
1e0c67c306 LF: Add RoundingMode and BigNumeric in Haskell Ast (#9215)
e492603808 More manual list fusion in stdlib (#9217)
90aa84193f tuesday notif: newlines (#9213)
51c5eac66e LF: Add BigNumeric to archive proto (#9210)
615f4e4bae Support passing sandbox port via --sandbox-option --port (#9208)
349e0a0bef LF: fix package name in LF 1.12 proto snapshot (#9211)
b7aedb477b Remove contract arguments from localContracts (#9207)
c1cf2756a8 add rollback node to transactions (#9178)
2551b2d615 LF: Add BigNumeric and Rounding to Scala AST (#9205)
be4dbd971d update compat versions for 1.11.1 (#9172)
0f9ab428ee kvutils: Remove ledger dump functionality (#9204)
f89aa294e5 damlc pkg: use cache for already present packages (#9193)
7a41ee6ce0 Cache contract fetches in speedy (#9192)
644561fe23 Bump jinja2 from 2.11.2 to 2.11.3 in /docs/scripts (#9198)
bf76e3c84a update NOTICES file (#9201)
4d90d02740 LF: drop useless field arity in Ast Builtin (#9194)
a93f89b6f8 Factor out an Oracle test fixture (#9191)
51f495e758 KVL-203 Share participant state integration test harness (#9143)
440eead6c4 Make LedgerDao a pure interface (#9186)
617e6f58e8 update NOTICES file (#9190)
587a9c95d0 Recognize (some) exerciseByKey commands in Daml script dumps (#9185)
ad94c47605 ci/docker: sync with hub (#9184)
081623b493 fix: bad package id validation (#9183)
f0c8b1240d Integrate non-repudiation into HTTP JSON API (#9180)
dbd8806848 Distinguish all four types of submits (#9179)
8c64f120da pkgid data deps (#9153)
d5ca22815d Recognize simple createAndExercise commands (#9154)
6c9679f511 Add documentation for the Daml Profiler (#9174)
a6da995ecb use representational variance for generated Scala classes (#8879)
47e0a223e7 rotate release duty after 1.12.0-snapshot.20210316.6523.0.b382fc45 (#9161)
1ea00976f0 skip compat tests on release (#9169)
c26c349c8b Generate exception instances from syntax. (#9140)
9d175ce547 Document profiling internals (#9168)
5e97c573e0 Add a test for create and exercise events in the profiler (#9151)
75f7688cae Expose unhandled exception values and types (#9158)
e4d5799907 weekly release (#9167)
370f290d3d Handle createAndExecuteCmd (#9150)
af0cb30828 Release SDK 1.11.1 (#9162)
```
Changelog:
```
- [Daml Assistant] The sandbox port can now also be configured via
`--sandbox-option=--port=12345` instead of `--sandbox-port`. Other
tools like Navigator, the JSON API and Daml Script will pick up the
modified port automatically.
- [JSON API] The JAR for the HTTP JSON API is no longer published on
Maven, use the fat JAR as indicated in the documentation.
- [Daml Profiler] Daml Connect EE now includes a profiler which can be
used to profile Daml execution time.
- [Scala Codegen] Type parameters used in GenMap keys are now supported.
See `issue #8879 <https://github.com/digital-asset/daml/pull/8879>`__.
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@sofiafaro-da is in charge of this release.
Commit log:
```
b382fc45ac update areas to alert @S11001001 of changes (#9156)
bbd239b0d3 Unstringify profiler labels (#9147)
3870f84534 Drop profiler from Sandbox CE (#9152)
691edeacf2 ci: fix cache cleanup (#9137)
98c2651998 Support fetching SDK EE tarball in the assistant (#9146)
9b2158508b Add new variant to Value.scala for builtin-exceptions. (#9084)
75cec502ac Add tests for unmangleLenient (#9148)
1df2270cc9 typo (#9145)
6018697fb4 Make async commit configurable and add "LOCAL" async commit option (#9144)
ab90c437a7 update compat versions for 1.11.1-snapshot.20210312.6429.1.7cd6380e (#9135)
498fcc66b4 KV specific LF library (#9100)
c327476da9 Upgrade msys packages (#9139)
089a11443c release job: remove extra bash-lib (#9136)
b0948b417b release 1.11.1-snapshot, take n+1 (#9134)
5dddf0aead Add missing checkout step (#9133)
7669d8c88c daml pkg: split installation of deps and package db inititialization (#9056)
d5e0d6ca00 Release 1.11.1 snapshot (#9130)
7430590c1d Only track cids referenced in root events (#9131)
d4bdb12862 Add pruning stabilization to ledger API changelog (#9126)
ab7e7d9d94 Ignore fetch and lookup-by-key nodes for state update comparison [KVL-854] (#9070)
208f6c1aab Fix javadocs & sources in daml-sdk-head (#9125)
9320a213db [Integration Kit, Ledger API] Remove hint about early access for participant pruning. (#9120)
1c5e64cb82 Fix release notes link in docs (#9122)
ed746976a3 Apply new logo to create-daml-app template (#9105)
7370313bfa Release yet another snapshot (#9118)
```
Changelog:
```
- [Daml Assistant] The assistant now supports an `artifactory-api-key`
field in `daml-config.yaml`. If you have a license for Daml Connect
EE you can specify this and the assistant will automatically fetch
the EE edition which provides additional functionality.
[Integration Kit, Ledger API] Remove hint about early access for participant pruning. It's stable now.
[SDK Assistant] New Daml logo on create-daml-app template
```
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Bumping the snapshot release number as the fix was on the release
process on main (i.e. not change to the target branch/commit).
CHANGELOG_BEGIN
CHANGELOG_END
* release 1.11.0-snapshot.20210309.6463.0.f7abca91
This PR has been created by a script, which is not very smart
and does not have all the context. Please do double-check that
the version prefix is correct before merging.
@S11001001 is in charge of this release.
Commit log:
```
f7abca919f fix readme in create-daml-app: add npm install (#9060)
e04bd91eda Move Daml Profiler to EE version of sandbox/sandbox-classic (#9054)
7859bc13e1 inline help: scripts JSON API clarification (#9057)
642385a692 Upgrade elliptic dependency to address vulnerabilities (#9055)
f2b9f118cf Check that only the command signed payload goes through (#9053)
5197374cab Build SDK EE tarball (#9049)
1bec2116ba Add tests for client bindings (#9036)
27fd9327f6 Disclose transactions in the flat transactions stream only to the stakeholders - test case [DPP-276] (#9028)
45b33757b2 Introduce a helper to limit signing to commands (#9050)
0c4d8ac19c Speedy: rollback within exercise (#9040)
caa023b72e ci/cron/check: remove dade-assist calls (#9048)
85f0965ab9 LF: fix archive decoder for GREATER_DATE (#9047)
121534c54d ci/cron/check: low-hanging perf improvement (#9042)
fb0bd82740 update NOTICES file (#9045)
4fd42a6772 reduce noise on daily tests (#9039)
5d88c08832 Bump ghcide (#9041)
41d62ea5c0 send CI failures on separate channel (#9038)
e26d006f6f slack: reduce @here mentions (#9037)
ae110361f0 Add more shared infrastructure for non-repudiation testing (#9035)
75140744aa Speedy: building transactions with rollback (#8983)
98410e7c7f update compat versions for 1.11.0-snapshot.20210304.6422.0.d3d5042a (#9032)
0f1d2a9c47 Upgrade akka to 2.6.13 (#9026)
b1a59d0c70 Release SDK 1.11.0 RC 2 (#9031)
41e049339b Fix --extract option in ledger api test tool (#9029)
0281b442b8 -Werror all the things (#9027)
2c08586d33 Expose Daml stacktraces for Daml Script errors (#9025)
c7ee410fa3 Use DA.Map in triggers if available (#9023)
d347934db3 Make non-repudiation PostgreSQL back-end certificate adding idempotent (#9024)
eaf7fb6f63 Add non-repudiation API (#8982)
5a3c95251c Document latency of Windows ad-hoc machine software installation (#9022)
3758b57a87 pkg deps: Move all dalfs to package dependency directory (#9002)
f70d3391ef update compat versions for 1.10.2 (#9016)
e5421288d9 Improve errors on duplicate record field names (#9010)
65fbcfe8e9 Move execution of commands out of Runner (#9009)
2688ad6f0d autorelease: improved PR message (#9008)
185e91646d Release 1.10.2 (#9006)
4ed3f1a6de Cut a snapshot release for the 1.8.0 special patch (#8998)
ba1456fd5a rotate release duty after 1.11.0-snapshot.20210302.6414.0.72870630 (#8992)
259bf682c4 compat: fix killing stale sandbox(es) (#9003)
32dc8b007b Include stack traces for all ScriptF commands (#8999)
1525957c7a release 1.11.0-snapshot.20210302.6414.0.72870630 (#8991)
926949e503 Use single party submit/submitTree where appropriate (#8995)
```
Changelog:
```
- [Daml Compiler] Fix a bug where passing `--ghc-option=-Werror` also
produced errors for warnings produced by `-Wmissing-signatures` even
if the user did not explicitly enable this.
- [Daml Script] When running Daml Script on the command line you will
now see a Daml stacktrace on failures to interact with the ledger
which makes it significantly easier to track down which of the calls
fails. By default, you will only get the callsite of functions like
`submit`. To extend the stack trace, add `HasCallStack` constraints
to functions and those will also be included.
- [Triggers] The trigger library now uses `DA.Map` instead of the
deprecated `DA.Next.Map` if the targeted Daml-LF version supports it.
This is a breaking change: Code that interfaced with the triggers
library using `DA.Next.Map`, e.g. with
`Daml.Trigger.getCommandsInFlight` or `Daml.Trigger.Assert.testRule`,
will need to be changed to use `DA.Map` instead.
```
CHANGELOG_BEGIN
CHANGELOG_END
* 1.12 not 1.11
changelog_begin
changelog_end
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* cut a snapshot release for the 1.8.0 special patch
CHANGELOG_BEGIN
CHANGELOG_END
* update for #9000
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@stefanobaghino-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@garyverhaegen-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@hurryabit is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* release 1.9.0-snapshot.20210119.6103.0.cdcf090b
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@nickchapman-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
* Bump version, changing prefix from 1.9 to 1.10
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Release yet another 1.9.0 snapshot
This includes #8490 which fixes triggers with LF 1.11. Since LF 1.11
is mentioned very prominently in our release notes, we don’t want to
break triggers there.
changelog_begin
changelog_end
* Release: Update to commit 9ed787cb3d.
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* release 1.9.0-snapshot.20210105.5984.0.c68ba110
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@robin-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
* Bump commit to include #8402
changelog_begin
changelog_end
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
This is not intended for general consumption.
This reverts the following PRs, in reverse order:
- #8082
- #8146
- #8153
- #8180
- #8194
CHANGELOG_BEGIN
- [Integration Kit] Reverted a fix to the kvutils input state that
caused performance degradation on some ledgers. This is a temporary
measure that we do not expect to be used outside of very specific
circumstances, as it can trade correctness for performance if the
semantics are not completely understood.
CHANGELOG_END
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@stefanobaghino-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@cocreature is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@garyverhaegen-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
There is no other way to test this code than to make a release. This is
expected to be identical to 1.7.0-snapshot.20201103.5565.0.e75d42dd,
with the version number being the only difference.
CHANGELOG_BEGIN
CHANGELOG_END
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@aherrmann-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@nickchapman-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* Release another 1.6.0 snapshot
This includes the backported timeout bump from 5cb20c1145.
changelog_begin
changelog_end
* add gsg fix
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@hurryabit is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
* release 1.6.0-snapshot.20200929.5303.0.f1e58206
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@sofiafaro-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
* Bump release commit
changelog_begin
changelog_end
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@S11001001 is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Note that following the changed process for release notes which now
just point to the blog, this does not include an update to the release
notes in the rst file.
changelog_begin
changelog_end
* release 1.5.0-snapshot.20200901.5116.0.4460cb5e
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@rohanjr is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
* release 1.5.0-snapshot.20200902.5118.0.2b3cf1b3
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@leo-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@garyverhaegen-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
This PR has been created by a script, which is not very smart and does not have all the context. Please do double-check that the version prefix is correct before merging.
@garyverhaegen-da is in charge of this release.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Azure Pipelines DAML Build <support@digitalasset.com>
* Release 1.3.0 snapshot
This includes the fix for the indexer, a fix for the stackoverflow and
unpublishes daml-on-sql from GH releases.
changelog_begin
changelog_end
* Bump commit
changelog_begin
changelog_end
Note for reviewers: This is a hotfix for a snapshot. I’m not going to
into why we need this here. As for the version number we decided that
the least confusing option here is the following:
Use the sha and number of commits as usual but use the date of the
original snapshot that we are patching here.
changelog_begin
changelog_end
trigger all releases from master
The 1.1.0 release went wrong and we had to trash it and release 1.1.1
instead. This is an attempt at identifying and correcting the root
cause behind that incident.
To understand the situation, we need to know how releases worked before
1.0. We had a one-line file called `LATEST` that specifies the git SHA and
version tag for the latest release. A change to that file triggered a
release with the specified release tag, built from the source tree of
the specified commit. The `LATEST` file looked something like:
```
f050da78c9 1.0.0-snapshot.20200411.3905.0.f050da78
```
To mark a release as stable, we would change it to look like this:
```
f050da78c9 1.0.0
```
i.e. simply drop the `-snapshot...` suffix. Even though the commit (and
thus the entire source tree we build from) is the same, we would need to
rebuild almost all of our release artifacts, as they embed the version
tag in various places and ways. That worked well as long as we could
assume we were doing trunk-based development, i.e. all releases would
always come from the same (`master`) branch.
When we released 1.0, and started work on 1.1, we had a few bug reports
for 1.0 that we decided should be resolved in a point release. We
decided that the best way to handle that would be to have a branch
starting on the release commit for 1.0, and then backport patches from
`master` to that branch. We adapted our build process to also watch the
`release/1.0.x` branch and, in particular, trigger a new release build if
the `LATEST` file in that branch changed. That worked well.
The plan going forward was to keep doing regular snapshot releases from
the `master` branch, and create support, point releases ("patch" releases
in semver) from dedicated branches.
On April 30, we made a snapshot release as an RC for 1.1.0, by changing
the `LATEST` file in the `master` branch. That release was built on commit
681c862d. On May 6, we decided to take a new snapshot as the RC for
1.1.0; we changed `LATEST` in `master` to designate 7e448d81 as the new
latest release.
On May 11, we noticed an issue that broke our builds. Without going into
details, an external artifact we depend on had changed in incompatible
ways. After fixing that on `master`, we reasoned that this would also
break the build of the final 1.1.0 release if we just tried to build
7e448d81 again. But as the target release date was May 13, we did not
want to take a new snapshot after that fix, as that would have included
one more week of work in the release, and given us no time to test it.
So we did what we did for the 1.0 branch, as it had worked well: we
created a branch that forked from `master` at commit 7e448d81 and called
it `release/1.1.x`, then cherry-picked the one fix to our build process to
work around the broken download. When the time came to make the final
1.1.0 build on May 13, we naturally picked the `LATEST` file from the
`release/1.1.x` branch and dropped the `-snapshot...` suffix. Importantly,
we did not need to update the target commit to include the "broken
download" fix as, in the meantime, the internet had fixed itself, and we
thus reasoned we should go for the exact code of the RC rather than
include an unnecessary, albeit seemingly harmless, change.
Everything went well with the release process. Tests went well too. Then
we got a report that an application that worked against the latest RC
broke with the final 1.1.0. The issue was that we had built the wrong
commit: by branching off at the point of the _target_ commit for the
latest snapshot, we did not have the change to the `LATEST` file that
designated that commit as the target. So the `LATEST` file in
`release/1.1.x` was still pointing to 681c862d.
I believe the root cause for this issue is the fact that we have
scattered our release process over multiple branches, meaning there is
no linear history of what was released and we are relying on people
being able to mentally manage multiple timelines. Therefore, I propose
to fix our release process so this should not happen again by
linearizing the release process, i.e. getting back to a situation where
all releases are made from a single branch, `master`.
Because we do want to be able to release _for_ multiple release branches
(to provide backports and bugfixes), we still need some way to
accommodate that. Having a single `LATEST` file in the same format as
before would not really work well: keeping track of interleaved release
streams on a single file would not really be easier than keeping track
of multiple branches.
My proposed solution is to instead have a multiline LATEST file, so that
all the release branch "tips" can be observed at the same time, and, as
long as we take care to only advance one release branch at a time, we
can easily keep track of each of them. This is what this PR does.
This required a few changes to our release process. Most notably:
- Obviously, as this is the main point of this PR, the build process has
once again been restricted to only trigger new releases from the
`master` branch.
- As our CI machinery cannot easily be made to produce multiple releases
from a single build, the `check_for_release` step will only recognize
a commit as a release trigger if it changes a single line in the
`LATEST` file. This restriction comes in addition to the existing one
that a release commit is only allowed to change either just the
`LATEST` file or both the `LATEST` and
`docs/source/support/release-notes.rst` files.
- The docs publication process has been changed to update _all_
published versions to display the _latest_ release notes page. This
means that the release notes page will always show you all published
versions, regardless of which version of the documentation you're
looking at. This also means that interleaving release notes correctly on
that page is a manual exercise.
- As per the intention of the new process, the `LATEST` file has been
updated to contained all existing post-1.0 stable releases. It should
also include all existing snapshot releases should we have more than one
at a time (say, should we discover an issue with 1.1.1 that required us
to work on a 1.1.2).
- The `release.sh` script has been dramatically simplified as I felt it
was trying to do too much and porting its existing functionality to a
multi-line `LATEST` file would be too hard.
CHANGELOG_BEGIN
CHANGELOG_END
This is not intended to be the next stable release, I just want to
test the Bintray removal and publishing the protobufs to github releases.
changelog_begin
changelog_end
Let's see how far we get with #4745.
It's a bit of a shame I can't retry the same commit multiple times.
Maybe I should have accounted for that in the version format...
CHANGELOG_BEGIN
CHANGELOG_END
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
