2020-03-27 03:26:10 +03:00
|
|
|
# Copyright (c) 2020 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
|
2019-04-04 11:33:38 +03:00
|
|
|
# SPDX-License-Identifier: Apache-2.0
|
|
|
|
|
|
2020-01-16 17:55:32 +03:00
|
|
|
load("@build_bazel_rules_nodejs//:index.bzl", "nodejs_binary")
|
2019-06-28 13:55:31 +03:00
|
|
|
load("@os_info//:os_info.bzl", "is_linux", "is_windows")
|
2020-03-19 02:06:55 +03:00
|
|
|
load("//rules_daml:daml.bzl", "daml_build_test", "daml_compile", "daml_test")
|
2019-06-18 12:56:06 +03:00
|
|
|
load("@bazel_tools//tools/build_defs/pkg:pkg.bzl", "pkg_tar")
|
introduce new release process (#4513)
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-25 19:01:23 +03:00
|
|
|
load("@build_environment//:configuration.bzl", "mvn_version", "sdk_version")
|
2019-04-04 11:33:38 +03:00
|
|
|
|
|
|
|
|
nodejs_binary(
|
2019-04-12 14:10:16 +03:00
|
|
|
name = "grunt",
|
|
|
|
|
data = [
|
|
|
|
|
"@npm//grunt-cli",
|
|
|
|
|
"@npm//matchdep",
|
|
|
|
|
],
|
2019-07-05 17:04:47 +03:00
|
|
|
entry_point = "@npm//:node_modules/grunt-cli/bin/grunt",
|
2019-04-04 11:33:38 +03:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
genrule(
|
|
|
|
|
name = "theme",
|
2019-04-12 14:10:16 +03:00
|
|
|
srcs = glob(
|
|
|
|
|
["theme/**"],
|
|
|
|
|
exclude = [
|
|
|
|
|
"theme/bower_components/**",
|
|
|
|
|
"theme/node_modules/**",
|
|
|
|
|
"theme/da_theme/**",
|
|
|
|
|
],
|
|
|
|
|
) + [
|
2020-01-16 17:55:32 +03:00
|
|
|
# we need to list all the transitive dependencies here because of https://github.com/bazelbuild/rules_nodejs/issues/1553
|
|
|
|
|
"@npm//acorn",
|
|
|
|
|
"@npm//acorn-node",
|
|
|
|
|
"@npm//acorn-walk",
|
|
|
|
|
"@npm//ansi-regex",
|
|
|
|
|
"@npm//ansi-styles",
|
|
|
|
|
"@npm//anymatch",
|
|
|
|
|
"@npm//assert",
|
|
|
|
|
"@npm//async",
|
|
|
|
|
"@npm//async-each",
|
|
|
|
|
"@npm//balanced-match",
|
|
|
|
|
"@npm//basic-auth",
|
|
|
|
|
"@npm//batch",
|
|
|
|
|
"@npm//binary-extensions",
|
|
|
|
|
"@npm//body",
|
|
|
|
|
"@npm//brace-expansion",
|
2019-04-12 14:10:16 +03:00
|
|
|
"@npm//browserify",
|
2020-01-16 17:55:32 +03:00
|
|
|
"@npm//browserify-cache-api",
|
|
|
|
|
"@npm//browserify-incremental",
|
|
|
|
|
"@npm//browserify-zlib",
|
|
|
|
|
"@npm//browser-pack",
|
|
|
|
|
"@npm//browser-resolve",
|
|
|
|
|
"@npm//buffer",
|
|
|
|
|
"@npm//buffer-from",
|
Upgrade rules_nodejs to version 1.6.0 (#5539)
* Upgrade rules_nodejs to version 1.6.0
closes #5367
This includes the fixes for the issues in jest that we’ve been seeing.
changelog_begin
changelog_end
* Fix eslint rules
* A bit of progress
* Try to add LinkablePackageInfo (doesn’t seem to work yet)
* Add rootDirs
* revert da_ts_library
* da_ts_library: add LinkablePackageInfo info
* Remove react hook workaround
Since rules_nodejs 1.6.0 this fails with the following error:
```
● Test suite failed to run
Configuration error:
Could not locate module react mapped as:
/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js.
Please check your configuration for these entries:
{
"moduleNameMapper": {
"/^react$/": "/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js"
},
"resolver": null
}
49 | // like a promis without being one.
50 | /* eslint-disable @typescript-eslint/no-floating-promises */
> 51 | var react_1 = __importStar(require("react"));
| ^
52 | var react_hooks_1 = require("@testing-library/react-hooks");
53 | var index_1 = __importStar(require("./index"));
54 | var events_1 = require("events");
at createNoMappedModuleFoundError (../../../../../../../../../../../node_modules/jest-resolve/build/index.js:501:17)
at Object.<anonymous> (index.test.js:51:28)
Test Suites: 1 failed, 1 total
Tests: 0 total
Snapshots: 0 total
Time: 1.88s
Ran all test suites within paths "language-support/ts/daml-react/DamlLedger.d.ts", "language-support/ts/daml-react/DamlLedger.js", "language-support/ts/daml-react/context.d.ts", "language-support/ts/daml-react/context.js", "language-support/ts/daml-react/hooks.d.ts", "language-support/ts/daml-react/hooks.js", "language-support/ts/daml-react/index.d.ts", "language-support/ts/daml-react/index.js", "language-support/ts/daml-react/index.test.d.ts", "language-support/ts/daml-react/index.test.js".
=
```
* rootDirs is not needed for tsc
This is only required for ts_project
* Update yarn Bazel packages
* docs/theme add missing dependencies
* Remove unused attribute module_root
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-04-17 15:03:33 +03:00
|
|
|
"@npm//bytes",
|
2020-01-16 17:55:32 +03:00
|
|
|
"@npm//cached-path-relative",
|
|
|
|
|
"@npm//chalk",
|
|
|
|
|
"@npm//chokidar",
|
|
|
|
|
"@npm//coffeescript",
|
|
|
|
|
"@npm//color-convert",
|
|
|
|
|
"@npm//colors",
|
|
|
|
|
"@npm//combine-source-map",
|
|
|
|
|
"@npm//concat-map",
|
|
|
|
|
"@npm//concat-stream",
|
|
|
|
|
"@npm//connect",
|
|
|
|
|
"@npm//connect-livereload",
|
|
|
|
|
"@npm//console-browserify",
|
|
|
|
|
"@npm//constants-browserify",
|
|
|
|
|
"@npm//continuable-cache",
|
|
|
|
|
"@npm//convert-source-map",
|
|
|
|
|
"@npm//core-util-is",
|
|
|
|
|
"@npm//cross-spawn",
|
|
|
|
|
"@npm//crypto-browserify",
|
|
|
|
|
"@npm//dargs",
|
|
|
|
|
"@npm//dash-ast",
|
|
|
|
|
"@npm//dateformat",
|
|
|
|
|
"@npm//defined",
|
|
|
|
|
"@npm//depd",
|
|
|
|
|
"@npm//deps-sort",
|
|
|
|
|
"@npm//destroy",
|
|
|
|
|
"@npm//detective",
|
|
|
|
|
"@npm//domain-browser",
|
|
|
|
|
"@npm//duplexer",
|
|
|
|
|
"@npm//duplexer2",
|
|
|
|
|
"@npm//ee-first",
|
|
|
|
|
"@npm//encodeurl",
|
|
|
|
|
"@npm//error",
|
|
|
|
|
"@npm//escape-html",
|
|
|
|
|
"@npm//escape-string-regexp",
|
|
|
|
|
"@npm//etag",
|
|
|
|
|
"@npm//eventemitter2",
|
|
|
|
|
"@npm//events",
|
|
|
|
|
"@npm//exit",
|
2020-06-11 20:36:40 +03:00
|
|
|
"@npm//fast-safe-stringify",
|
2020-01-16 17:55:32 +03:00
|
|
|
"@npm//faye-websocket",
|
|
|
|
|
"@npm//figures",
|
|
|
|
|
"@npm//file-sync-cmp",
|
Upgrade rules_nodejs to version 1.6.0 (#5539)
* Upgrade rules_nodejs to version 1.6.0
closes #5367
This includes the fixes for the issues in jest that we’ve been seeing.
changelog_begin
changelog_end
* Fix eslint rules
* A bit of progress
* Try to add LinkablePackageInfo (doesn’t seem to work yet)
* Add rootDirs
* revert da_ts_library
* da_ts_library: add LinkablePackageInfo info
* Remove react hook workaround
Since rules_nodejs 1.6.0 this fails with the following error:
```
● Test suite failed to run
Configuration error:
Could not locate module react mapped as:
/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js.
Please check your configuration for these entries:
{
"moduleNameMapper": {
"/^react$/": "/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js"
},
"resolver": null
}
49 | // like a promis without being one.
50 | /* eslint-disable @typescript-eslint/no-floating-promises */
> 51 | var react_1 = __importStar(require("react"));
| ^
52 | var react_hooks_1 = require("@testing-library/react-hooks");
53 | var index_1 = __importStar(require("./index"));
54 | var events_1 = require("events");
at createNoMappedModuleFoundError (../../../../../../../../../../../node_modules/jest-resolve/build/index.js:501:17)
at Object.<anonymous> (index.test.js:51:28)
Test Suites: 1 failed, 1 total
Tests: 0 total
Snapshots: 0 total
Time: 1.88s
Ran all test suites within paths "language-support/ts/daml-react/DamlLedger.d.ts", "language-support/ts/daml-react/DamlLedger.js", "language-support/ts/daml-react/context.d.ts", "language-support/ts/daml-react/context.js", "language-support/ts/daml-react/hooks.d.ts", "language-support/ts/daml-react/hooks.js", "language-support/ts/daml-react/index.d.ts", "language-support/ts/daml-react/index.js", "language-support/ts/daml-react/index.test.d.ts", "language-support/ts/daml-react/index.test.js".
=
```
* rootDirs is not needed for tsc
This is only required for ts_project
* Update yarn Bazel packages
* docs/theme add missing dependencies
* Remove unused attribute module_root
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-04-17 15:03:33 +03:00
|
|
|
"@npm//finalhandler",
|
2020-01-16 17:55:32 +03:00
|
|
|
"@npm//findup-sync",
|
|
|
|
|
"@npm//fresh",
|
|
|
|
|
"@npm//fs.realpath",
|
|
|
|
|
"@npm//function-bind",
|
|
|
|
|
"@npm//gaze",
|
|
|
|
|
"@npm//get-assigned-identifiers",
|
|
|
|
|
"@npm//getobject",
|
|
|
|
|
"@npm//glob",
|
|
|
|
|
"@npm//glob-parent",
|
|
|
|
|
"@npm//globule",
|
|
|
|
|
"@npm//graceful-fs",
|
|
|
|
|
"@npm//grunt",
|
2019-04-12 14:10:16 +03:00
|
|
|
"@npm//grunt-banner",
|
|
|
|
|
"@npm//grunt-browserify",
|
|
|
|
|
"@npm//grunt-contrib-clean",
|
|
|
|
|
"@npm//grunt-contrib-connect",
|
|
|
|
|
"@npm//grunt-contrib-copy",
|
|
|
|
|
"@npm//grunt-contrib-sass",
|
|
|
|
|
"@npm//grunt-contrib-uglify",
|
|
|
|
|
"@npm//grunt-contrib-watch",
|
|
|
|
|
"@npm//grunt-exec",
|
2020-01-16 17:55:32 +03:00
|
|
|
"@npm//grunt-legacy-log",
|
|
|
|
|
"@npm//grunt-legacy-log-utils",
|
|
|
|
|
"@npm//grunt-legacy-util",
|
2019-04-12 14:10:16 +03:00
|
|
|
"@npm//grunt-open",
|
2020-01-16 17:55:32 +03:00
|
|
|
"@npm//gzip-size",
|
|
|
|
|
"@npm//has",
|
|
|
|
|
"@npm//has-ansi",
|
|
|
|
|
"@npm//has-flag",
|
|
|
|
|
"@npm//hooker",
|
|
|
|
|
"@npm//htmlescape",
|
Upgrade rules_nodejs to version 1.6.0 (#5539)
* Upgrade rules_nodejs to version 1.6.0
closes #5367
This includes the fixes for the issues in jest that we’ve been seeing.
changelog_begin
changelog_end
* Fix eslint rules
* A bit of progress
* Try to add LinkablePackageInfo (doesn’t seem to work yet)
* Add rootDirs
* revert da_ts_library
* da_ts_library: add LinkablePackageInfo info
* Remove react hook workaround
Since rules_nodejs 1.6.0 this fails with the following error:
```
● Test suite failed to run
Configuration error:
Could not locate module react mapped as:
/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js.
Please check your configuration for these entries:
{
"moduleNameMapper": {
"/^react$/": "/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js"
},
"resolver": null
}
49 | // like a promis without being one.
50 | /* eslint-disable @typescript-eslint/no-floating-promises */
> 51 | var react_1 = __importStar(require("react"));
| ^
52 | var react_hooks_1 = require("@testing-library/react-hooks");
53 | var index_1 = __importStar(require("./index"));
54 | var events_1 = require("events");
at createNoMappedModuleFoundError (../../../../../../../../../../../node_modules/jest-resolve/build/index.js:501:17)
at Object.<anonymous> (index.test.js:51:28)
Test Suites: 1 failed, 1 total
Tests: 0 total
Snapshots: 0 total
Time: 1.88s
Ran all test suites within paths "language-support/ts/daml-react/DamlLedger.d.ts", "language-support/ts/daml-react/DamlLedger.js", "language-support/ts/daml-react/context.d.ts", "language-support/ts/daml-react/context.js", "language-support/ts/daml-react/hooks.d.ts", "language-support/ts/daml-react/hooks.js", "language-support/ts/daml-react/index.d.ts", "language-support/ts/daml-react/index.js", "language-support/ts/daml-react/index.test.d.ts", "language-support/ts/daml-react/index.test.js".
=
```
* rootDirs is not needed for tsc
This is only required for ts_project
* Update yarn Bazel packages
* docs/theme add missing dependencies
* Remove unused attribute module_root
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-04-17 15:03:33 +03:00
|
|
|
"@npm//http-errors",
|
2020-01-16 17:55:32 +03:00
|
|
|
"@npm//http-parser-js",
|
|
|
|
|
"@npm//https-browserify",
|
|
|
|
|
"@npm//iconv-lite",
|
|
|
|
|
"@npm//inflight",
|
|
|
|
|
"@npm//inherits",
|
|
|
|
|
"@npm//inline-source-map",
|
|
|
|
|
"@npm//insert-module-globals",
|
|
|
|
|
"@npm//is-binary-path",
|
|
|
|
|
"@npm//is-number-like",
|
|
|
|
|
"@npm//is-wsl",
|
|
|
|
|
"@npm//jsonparse",
|
|
|
|
|
"@npm//json-stable-stringify",
|
|
|
|
|
"@npm//JSONStream",
|
|
|
|
|
"@npm//js-yaml",
|
|
|
|
|
"@npm//labeled-stream-splicer",
|
|
|
|
|
"@npm//lodash",
|
|
|
|
|
"@npm//lodash.isfinite",
|
|
|
|
|
"@npm//lodash.memoize",
|
|
|
|
|
"@npm//lru-cache",
|
2019-04-12 14:10:16 +03:00
|
|
|
"@npm//matchdep",
|
2020-01-16 17:55:32 +03:00
|
|
|
"@npm//maxmin",
|
|
|
|
|
"@npm//mime-db",
|
|
|
|
|
"@npm//mime-types",
|
|
|
|
|
"@npm//minimatch",
|
|
|
|
|
"@npm//mkdirp",
|
|
|
|
|
"@npm//module-deps",
|
|
|
|
|
"@npm//morgan",
|
|
|
|
|
"@npm//node-http2",
|
|
|
|
|
"@npm//nopt",
|
|
|
|
|
"@npm//normalize-path",
|
|
|
|
|
"@npm//number-is-nan",
|
|
|
|
|
"@npm//object-assign",
|
|
|
|
|
"@npm//once",
|
|
|
|
|
"@npm//on-finished",
|
|
|
|
|
"@npm//on-headers",
|
|
|
|
|
"@npm//opn",
|
|
|
|
|
"@npm//os-browserify",
|
|
|
|
|
"@npm//parents",
|
|
|
|
|
"@npm//parseurl",
|
|
|
|
|
"@npm//path-browserify",
|
|
|
|
|
"@npm//path-dirname",
|
|
|
|
|
"@npm//path-is-absolute",
|
|
|
|
|
"@npm//path-platform",
|
|
|
|
|
"@npm//portscanner",
|
|
|
|
|
"@npm//pretty-bytes",
|
|
|
|
|
"@npm//process",
|
|
|
|
|
"@npm//process-nextick-args",
|
Upgrade rules_nodejs to version 1.6.0 (#5539)
* Upgrade rules_nodejs to version 1.6.0
closes #5367
This includes the fixes for the issues in jest that we’ve been seeing.
changelog_begin
changelog_end
* Fix eslint rules
* A bit of progress
* Try to add LinkablePackageInfo (doesn’t seem to work yet)
* Add rootDirs
* revert da_ts_library
* da_ts_library: add LinkablePackageInfo info
* Remove react hook workaround
Since rules_nodejs 1.6.0 this fails with the following error:
```
● Test suite failed to run
Configuration error:
Could not locate module react mapped as:
/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js.
Please check your configuration for these entries:
{
"moduleNameMapper": {
"/^react$/": "/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js"
},
"resolver": null
}
49 | // like a promis without being one.
50 | /* eslint-disable @typescript-eslint/no-floating-promises */
> 51 | var react_1 = __importStar(require("react"));
| ^
52 | var react_hooks_1 = require("@testing-library/react-hooks");
53 | var index_1 = __importStar(require("./index"));
54 | var events_1 = require("events");
at createNoMappedModuleFoundError (../../../../../../../../../../../node_modules/jest-resolve/build/index.js:501:17)
at Object.<anonymous> (index.test.js:51:28)
Test Suites: 1 failed, 1 total
Tests: 0 total
Snapshots: 0 total
Time: 1.88s
Ran all test suites within paths "language-support/ts/daml-react/DamlLedger.d.ts", "language-support/ts/daml-react/DamlLedger.js", "language-support/ts/daml-react/context.d.ts", "language-support/ts/daml-react/context.js", "language-support/ts/daml-react/hooks.d.ts", "language-support/ts/daml-react/hooks.js", "language-support/ts/daml-react/index.d.ts", "language-support/ts/daml-react/index.js", "language-support/ts/daml-react/index.test.d.ts", "language-support/ts/daml-react/index.test.js".
=
```
* rootDirs is not needed for tsc
This is only required for ts_project
* Update yarn Bazel packages
* docs/theme add missing dependencies
* Remove unused attribute module_root
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-04-17 15:03:33 +03:00
|
|
|
"@npm//punycode",
|
2020-01-16 17:55:32 +03:00
|
|
|
"@npm//querystring-es3",
|
Upgrade rules_nodejs to version 1.6.0 (#5539)
* Upgrade rules_nodejs to version 1.6.0
closes #5367
This includes the fixes for the issues in jest that we’ve been seeing.
changelog_begin
changelog_end
* Fix eslint rules
* A bit of progress
* Try to add LinkablePackageInfo (doesn’t seem to work yet)
* Add rootDirs
* revert da_ts_library
* da_ts_library: add LinkablePackageInfo info
* Remove react hook workaround
Since rules_nodejs 1.6.0 this fails with the following error:
```
● Test suite failed to run
Configuration error:
Could not locate module react mapped as:
/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js.
Please check your configuration for these entries:
{
"moduleNameMapper": {
"/^react$/": "/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js"
},
"resolver": null
}
49 | // like a promis without being one.
50 | /* eslint-disable @typescript-eslint/no-floating-promises */
> 51 | var react_1 = __importStar(require("react"));
| ^
52 | var react_hooks_1 = require("@testing-library/react-hooks");
53 | var index_1 = __importStar(require("./index"));
54 | var events_1 = require("events");
at createNoMappedModuleFoundError (../../../../../../../../../../../node_modules/jest-resolve/build/index.js:501:17)
at Object.<anonymous> (index.test.js:51:28)
Test Suites: 1 failed, 1 total
Tests: 0 total
Snapshots: 0 total
Time: 1.88s
Ran all test suites within paths "language-support/ts/daml-react/DamlLedger.d.ts", "language-support/ts/daml-react/DamlLedger.js", "language-support/ts/daml-react/context.d.ts", "language-support/ts/daml-react/context.js", "language-support/ts/daml-react/hooks.d.ts", "language-support/ts/daml-react/hooks.js", "language-support/ts/daml-react/index.d.ts", "language-support/ts/daml-react/index.js", "language-support/ts/daml-react/index.test.d.ts", "language-support/ts/daml-react/index.test.js".
=
```
* rootDirs is not needed for tsc
This is only required for ts_project
* Update yarn Bazel packages
* docs/theme add missing dependencies
* Remove unused attribute module_root
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-04-17 15:03:33 +03:00
|
|
|
"@npm//raw-body",
|
2020-01-16 17:55:32 +03:00
|
|
|
"@npm//readable-stream",
|
|
|
|
|
"@npm//readdirp",
|
|
|
|
|
"@npm//read-only-stream",
|
|
|
|
|
"@npm//remove-trailing-separator",
|
|
|
|
|
"@npm//rimraf",
|
|
|
|
|
"@npm//safe-buffer",
|
|
|
|
|
"@npm//safe-json-parse",
|
|
|
|
|
"@npm//safer-buffer",
|
Upgrade rules_nodejs to version 1.6.0 (#5539)
* Upgrade rules_nodejs to version 1.6.0
closes #5367
This includes the fixes for the issues in jest that we’ve been seeing.
changelog_begin
changelog_end
* Fix eslint rules
* A bit of progress
* Try to add LinkablePackageInfo (doesn’t seem to work yet)
* Add rootDirs
* revert da_ts_library
* da_ts_library: add LinkablePackageInfo info
* Remove react hook workaround
Since rules_nodejs 1.6.0 this fails with the following error:
```
● Test suite failed to run
Configuration error:
Could not locate module react mapped as:
/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js.
Please check your configuration for these entries:
{
"moduleNameMapper": {
"/^react$/": "/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js"
},
"resolver": null
}
49 | // like a promis without being one.
50 | /* eslint-disable @typescript-eslint/no-floating-promises */
> 51 | var react_1 = __importStar(require("react"));
| ^
52 | var react_hooks_1 = require("@testing-library/react-hooks");
53 | var index_1 = __importStar(require("./index"));
54 | var events_1 = require("events");
at createNoMappedModuleFoundError (../../../../../../../../../../../node_modules/jest-resolve/build/index.js:501:17)
at Object.<anonymous> (index.test.js:51:28)
Test Suites: 1 failed, 1 total
Tests: 0 total
Snapshots: 0 total
Time: 1.88s
Ran all test suites within paths "language-support/ts/daml-react/DamlLedger.d.ts", "language-support/ts/daml-react/DamlLedger.js", "language-support/ts/daml-react/context.d.ts", "language-support/ts/daml-react/context.js", "language-support/ts/daml-react/hooks.d.ts", "language-support/ts/daml-react/hooks.js", "language-support/ts/daml-react/index.d.ts", "language-support/ts/daml-react/index.js", "language-support/ts/daml-react/index.test.d.ts", "language-support/ts/daml-react/index.test.js".
=
```
* rootDirs is not needed for tsc
This is only required for ts_project
* Update yarn Bazel packages
* docs/theme add missing dependencies
* Remove unused attribute module_root
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-04-17 15:03:33 +03:00
|
|
|
"@npm//send",
|
2020-01-16 17:55:32 +03:00
|
|
|
"@npm//serve-index",
|
Upgrade rules_nodejs to version 1.6.0 (#5539)
* Upgrade rules_nodejs to version 1.6.0
closes #5367
This includes the fixes for the issues in jest that we’ve been seeing.
changelog_begin
changelog_end
* Fix eslint rules
* A bit of progress
* Try to add LinkablePackageInfo (doesn’t seem to work yet)
* Add rootDirs
* revert da_ts_library
* da_ts_library: add LinkablePackageInfo info
* Remove react hook workaround
Since rules_nodejs 1.6.0 this fails with the following error:
```
● Test suite failed to run
Configuration error:
Could not locate module react mapped as:
/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js.
Please check your configuration for these entries:
{
"moduleNameMapper": {
"/^react$/": "/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js"
},
"resolver": null
}
49 | // like a promis without being one.
50 | /* eslint-disable @typescript-eslint/no-floating-promises */
> 51 | var react_1 = __importStar(require("react"));
| ^
52 | var react_hooks_1 = require("@testing-library/react-hooks");
53 | var index_1 = __importStar(require("./index"));
54 | var events_1 = require("events");
at createNoMappedModuleFoundError (../../../../../../../../../../../node_modules/jest-resolve/build/index.js:501:17)
at Object.<anonymous> (index.test.js:51:28)
Test Suites: 1 failed, 1 total
Tests: 0 total
Snapshots: 0 total
Time: 1.88s
Ran all test suites within paths "language-support/ts/daml-react/DamlLedger.d.ts", "language-support/ts/daml-react/DamlLedger.js", "language-support/ts/daml-react/context.d.ts", "language-support/ts/daml-react/context.js", "language-support/ts/daml-react/hooks.d.ts", "language-support/ts/daml-react/hooks.js", "language-support/ts/daml-react/index.d.ts", "language-support/ts/daml-react/index.js", "language-support/ts/daml-react/index.test.d.ts", "language-support/ts/daml-react/index.test.js".
=
```
* rootDirs is not needed for tsc
This is only required for ts_project
* Update yarn Bazel packages
* docs/theme add missing dependencies
* Remove unused attribute module_root
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2020-04-17 15:03:33 +03:00
|
|
|
"@npm//serve-static",
|
2020-06-11 20:36:40 +03:00
|
|
|
"@npm//setprototypeof",
|
2020-01-16 17:55:32 +03:00
|
|
|
"@npm//shasum",
|
2020-06-11 20:36:40 +03:00
|
|
|
"@npm//shasum-object",
|
2020-01-16 17:55:32 +03:00
|
|
|
"@npm//sprintf-js",
|
|
|
|
|
"@npm//statuses",
|
|
|
|
|
"@npm//stream-browserify",
|
|
|
|
|
"@npm//stream-combiner2",
|
|
|
|
|
"@npm//stream-http",
|
|
|
|
|
"@npm//stream-splicer",
|
|
|
|
|
"@npm//string_decoder",
|
|
|
|
|
"@npm//string-template",
|
|
|
|
|
"@npm//strip-ansi",
|
|
|
|
|
"@npm//supports-color",
|
|
|
|
|
"@npm//syntax-error",
|
|
|
|
|
"@npm//through",
|
|
|
|
|
"@npm//through2",
|
|
|
|
|
"@npm//timers-browserify",
|
|
|
|
|
"@npm//tiny-lr",
|
|
|
|
|
"@npm//tty-browserify",
|
|
|
|
|
"@npm//uglify-js",
|
|
|
|
|
"@npm//umd",
|
|
|
|
|
"@npm//undeclared-identifiers",
|
|
|
|
|
"@npm//underscore.string",
|
|
|
|
|
"@npm//unpipe",
|
|
|
|
|
"@npm//upath",
|
|
|
|
|
"@npm//uri-path",
|
|
|
|
|
"@npm//url",
|
|
|
|
|
"@npm//util",
|
|
|
|
|
"@npm//util-deprecate",
|
|
|
|
|
"@npm//utils-merge",
|
|
|
|
|
"@npm//vm-browserify",
|
|
|
|
|
"@npm//watchify",
|
|
|
|
|
"@npm//websocket-driver",
|
|
|
|
|
"@npm//websocket-extensions",
|
|
|
|
|
"@npm//wrappy",
|
|
|
|
|
"@npm//xtend",
|
2019-04-12 14:10:16 +03:00
|
|
|
],
|
2019-04-04 11:33:38 +03:00
|
|
|
outs = ["da_theme.tar.gz"],
|
|
|
|
|
cmd = """
|
|
|
|
|
cp -rL docs/theme theme
|
|
|
|
|
cd theme
|
|
|
|
|
|
|
|
|
|
# Make the node_modules available
|
|
|
|
|
ln -s ../external/npm/node_modules .
|
|
|
|
|
|
|
|
|
|
# Run sass and grunt
|
|
|
|
|
../$(location @sass_nix//:bin/sass) \
|
|
|
|
|
-I bower_components_static/bourbon/dist \
|
|
|
|
|
-I bower_components_static/neat/app/assets/stylesheets \
|
|
|
|
|
-I bower_components_static/font-awesome/scss \
|
|
|
|
|
-I bower_components_static/wyrm/sass \
|
|
|
|
|
--style compressed \
|
|
|
|
|
--sourcemap=none \
|
|
|
|
|
--update \
|
|
|
|
|
sass:da_theme/static/css
|
|
|
|
|
../$(location :grunt) build
|
|
|
|
|
|
2020-03-31 11:09:52 +03:00
|
|
|
tar c da_theme \
|
2019-04-09 19:59:37 +03:00
|
|
|
--owner=1000 \
|
|
|
|
|
--group=1000 \
|
2020-03-31 11:09:52 +03:00
|
|
|
--mtime=2000-01-01\ 00:00Z \
|
2019-04-09 19:59:37 +03:00
|
|
|
--no-acls \
|
|
|
|
|
--no-xattrs \
|
|
|
|
|
--no-selinux \
|
|
|
|
|
--sort=name \
|
2020-03-31 11:09:52 +03:00
|
|
|
| gzip -n > ../$(location da_theme.tar.gz)
|
2019-04-12 14:10:16 +03:00
|
|
|
""",
|
|
|
|
|
tools = [
|
|
|
|
|
":grunt",
|
|
|
|
|
"@sass_nix//:bin/sass",
|
|
|
|
|
],
|
2019-06-28 13:55:31 +03:00
|
|
|
) if not is_windows else None
|
2019-04-04 11:33:38 +03:00
|
|
|
|
|
|
|
|
genrule(
|
|
|
|
|
name = "sources",
|
2019-04-12 14:10:16 +03:00
|
|
|
srcs = glob(["source/**"]) + [
|
2020-04-23 18:12:48 +03:00
|
|
|
"//compiler/damlc:daml-base-rst.tar.gz",
|
2019-10-28 22:30:23 +03:00
|
|
|
"//triggers/daml:daml-trigger-rst-docs",
|
2019-11-21 14:59:09 +03:00
|
|
|
"//daml-script/daml:daml-script-rst-docs",
|
2019-04-12 14:10:16 +03:00
|
|
|
"//ledger-api/grpc-definitions:docs",
|
|
|
|
|
"//:LICENSE",
|
|
|
|
|
"//:NOTICES",
|
|
|
|
|
],
|
2019-04-04 11:33:38 +03:00
|
|
|
outs = ["source.tar.gz"],
|
|
|
|
|
cmd = """
|
|
|
|
|
cp -rL docs/source source
|
|
|
|
|
|
|
|
|
|
# Copy in Stdlib
|
2020-04-22 15:23:46 +03:00
|
|
|
mkdir -p source/daml/stdlib
|
2020-04-23 18:12:48 +03:00
|
|
|
tar xf $(location //compiler/damlc:daml-base-rst.tar.gz) \
|
2020-04-22 15:23:46 +03:00
|
|
|
--strip-components 1 \
|
|
|
|
|
-C source/daml/stdlib
|
2019-04-04 11:33:38 +03:00
|
|
|
|
2019-10-28 22:30:23 +03:00
|
|
|
# Copy in daml-trigger documentation
|
|
|
|
|
cp -rL $(location //triggers/daml:daml-trigger-rst-docs) source/triggers/trigger-docs.rst
|
|
|
|
|
|
2019-11-21 14:59:09 +03:00
|
|
|
# Copy in daml-script documentation
|
|
|
|
|
cp -rL $(location //daml-script/daml:daml-script-rst-docs) source/daml-script/daml-script-docs.rst
|
|
|
|
|
|
2019-04-04 11:33:38 +03:00
|
|
|
# Copy in Protobufs
|
2019-04-26 15:09:38 +03:00
|
|
|
cp -rL $(location //ledger-api/grpc-definitions:docs) source/app-dev/grpc/proto-docs.rst
|
2019-04-04 11:33:38 +03:00
|
|
|
|
|
|
|
|
# Copy in License and Notices
|
|
|
|
|
cp -L $(location //:LICENSE) source/LICENSE
|
|
|
|
|
cp -L $(location //:NOTICES) source/NOTICES
|
|
|
|
|
|
2020-03-31 11:09:52 +03:00
|
|
|
tar c source \
|
|
|
|
|
--owner=0 --group=0 --numeric-owner --mtime=2000-01-01\ 00:00Z --sort=name \
|
|
|
|
|
| gzip -n > $(location source.tar.gz)
|
2019-04-12 14:10:16 +03:00
|
|
|
""",
|
2019-04-04 11:33:38 +03:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
genrule(
|
|
|
|
|
name = "pdf-docs",
|
2019-04-12 14:10:16 +03:00
|
|
|
srcs = glob([
|
|
|
|
|
"configs/pdf/**",
|
|
|
|
|
"configs/static/pygments_daml_lexer.py",
|
2020-03-13 18:46:23 +03:00
|
|
|
"configs/static/typescript.py",
|
2019-04-12 14:10:16 +03:00
|
|
|
]) + [
|
|
|
|
|
":sources",
|
|
|
|
|
],
|
2019-04-04 11:33:38 +03:00
|
|
|
outs = ["DigitalAssetSDK.pdf"],
|
|
|
|
|
cmd = ("""
|
|
|
|
|
export LOCALE_ARCHIVE="$$PWD/$(location @glibc_locales//:locale-archive)"
|
|
|
|
|
""" if is_linux else "") + """
|
2019-04-08 12:10:03 +03:00
|
|
|
set -euo pipefail
|
2019-04-04 11:33:38 +03:00
|
|
|
# Set up tools
|
|
|
|
|
export PATH="$$( cd "$$(dirname "$(location @imagemagick_nix//:bin/convert)")" ; pwd -P )":$$PATH
|
|
|
|
|
|
|
|
|
|
# Copy files into the right structure and remove symlinks
|
|
|
|
|
tar -zxf $(location sources) -C .
|
|
|
|
|
cp -L docs/configs/pdf/index.rst source/
|
|
|
|
|
cp -L docs/configs/pdf/conf.py source/
|
|
|
|
|
cp -L docs/configs/pdf/logo.png source/
|
|
|
|
|
cp -rL docs/configs/static ./
|
|
|
|
|
|
|
|
|
|
# Build with Sphinx
|
introduce new release process (#4513)
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-25 19:01:23 +03:00
|
|
|
sed -i "s,__VERSION__,"{sdk}"," source/conf.py
|
2019-04-04 11:33:38 +03:00
|
|
|
export LC_ALL=en_US.UTF-8
|
|
|
|
|
export LANG=en_US.UTF-8
|
|
|
|
|
$(location @sphinx_nix//:bin/sphinx-build) -b latex source out
|
|
|
|
|
|
|
|
|
|
# Copy in fonts and build with lualatex
|
|
|
|
|
cp -L docs/configs/pdf/fonts/* out/
|
|
|
|
|
cd out
|
|
|
|
|
# run twice to generate all references properly (this is a latex thing...)
|
2019-04-08 12:10:03 +03:00
|
|
|
../$(location @texlive_nix//:bin/lualatex) -halt-on-error -interaction=batchmode --shell-escape *.tex
|
|
|
|
|
../$(location @texlive_nix//:bin/lualatex) -halt-on-error -interaction=batchmode --shell-escape *.tex
|
2020-05-11 12:47:54 +03:00
|
|
|
# NOTE, if you get errors of the following form:
|
|
|
|
|
#
|
|
|
|
|
# luaotfload | db : Font names database not found, generating new one.
|
|
|
|
|
# luaotfload | db : This can take several minutes; please be patient.
|
|
|
|
|
# luaotfload | db : Reload initiated (formats: otf,ttf,ttc); reason: "File not found: lmroman10-regular.".
|
|
|
|
|
#
|
|
|
|
|
# Then the error is most likely not font related. To debug the error
|
|
|
|
|
# run `bazel build` with `--sandbox_debug`, change into the sandbox
|
|
|
|
|
# directory and invoke lualatex from there. You will have to replicate
|
|
|
|
|
# the environment variable setup from above.
|
|
|
|
|
#
|
|
|
|
|
# In the past the following issues caused the error message above:
|
|
|
|
|
# - An update of sphinx in nixpkgs that had to be undone.
|
|
|
|
|
# - A missing texlive package that had to be added to the Nix derivation.
|
2019-04-04 11:33:38 +03:00
|
|
|
|
|
|
|
|
# Move output to target
|
introduce new release process (#4513)
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-25 19:01:23 +03:00
|
|
|
mv DigitalAssetSDK.pdf ../$(location DigitalAssetSDK.pdf)""".format(sdk = sdk_version),
|
2020-02-26 17:52:08 +03:00
|
|
|
tags = ["pdfdocs"],
|
2019-04-12 14:10:16 +03:00
|
|
|
tools =
|
|
|
|
|
[
|
|
|
|
|
"@texlive_nix//:bin/lualatex",
|
|
|
|
|
"@sphinx_nix//:bin/sphinx-build",
|
|
|
|
|
"@imagemagick_nix//:bin/convert",
|
|
|
|
|
] + (["@glibc_locales//:locale-archive"] if is_linux else []),
|
2019-06-28 13:55:31 +03:00
|
|
|
) if not is_windows else None
|
2019-04-04 11:33:38 +03:00
|
|
|
|
|
|
|
|
genrule(
|
|
|
|
|
name = "docs-no-pdf",
|
2019-04-12 14:10:16 +03:00
|
|
|
srcs = glob([
|
|
|
|
|
"configs/html/**",
|
|
|
|
|
"configs/static/pygments_daml_lexer.py",
|
2020-03-13 18:46:23 +03:00
|
|
|
"configs/static/typescript.py",
|
2019-04-12 14:10:16 +03:00
|
|
|
]) + [
|
|
|
|
|
":sources",
|
|
|
|
|
":theme",
|
2020-04-23 18:12:48 +03:00
|
|
|
"//compiler/damlc:daml-base-hoogle.txt",
|
2019-05-14 10:40:30 +03:00
|
|
|
"//language-support/java:javadoc",
|
2020-03-10 20:46:54 +03:00
|
|
|
"//language-support/ts/daml-react:docs",
|
|
|
|
|
"//language-support/ts/daml-ledger:docs",
|
|
|
|
|
"//language-support/ts/daml-types:docs",
|
2020-04-03 14:18:18 +03:00
|
|
|
"//templates:templates-tarball",
|
2020-04-17 19:42:24 +03:00
|
|
|
"//templates:create-daml-app-test-resources/messaging.patch",
|
|
|
|
|
"//templates:create-daml-app-test-resources/index.test.ts",
|
2020-05-05 20:35:30 +03:00
|
|
|
"@daml-cheat-sheet//:site",
|
2019-04-12 14:10:16 +03:00
|
|
|
],
|
2019-04-04 11:33:38 +03:00
|
|
|
outs = ["html-only.tar.gz"],
|
|
|
|
|
cmd = ("""
|
|
|
|
|
export LOCALE_ARCHIVE="$$PWD/$(location @glibc_locales//:locale-archive)"
|
|
|
|
|
""" if is_linux else "") + """
|
|
|
|
|
# Copy files into the right structure and remove symlinks
|
|
|
|
|
mkdir build
|
|
|
|
|
cp -rL docs build
|
|
|
|
|
tar -zxf $(location sources) -C build/docs
|
|
|
|
|
|
|
|
|
|
# Copy in theme
|
|
|
|
|
mkdir -p build/docs/theme
|
|
|
|
|
tar -zxf $(location :theme) -C build/docs/theme
|
|
|
|
|
|
2020-04-03 14:18:18 +03:00
|
|
|
# Copy templates for code snippets in getting started guide
|
2020-04-11 22:53:44 +03:00
|
|
|
CODE_DIR=$$PWD/build/docs/source/getting-started/code/
|
|
|
|
|
mkdir -p $$CODE_DIR
|
|
|
|
|
tar -zxf $(location //templates:templates-tarball) -C $$CODE_DIR
|
2020-04-17 19:42:24 +03:00
|
|
|
# Copy create-daml-app tests
|
|
|
|
|
mkdir $$CODE_DIR/testing
|
|
|
|
|
cp $(location //templates:create-daml-app-test-resources/index.test.ts) $$CODE_DIR/testing
|
2020-04-11 22:53:44 +03:00
|
|
|
# Apply patch for messaging feature (we only need the "after" state)
|
|
|
|
|
PATCH_TOOL=$$PWD/$(location @patch_dev_env//:patch)
|
2020-04-17 19:42:24 +03:00
|
|
|
MESSAGING_PATCH=$$PWD/$(location //templates:create-daml-app-test-resources/messaging.patch)
|
2020-04-11 22:53:44 +03:00
|
|
|
(cd $$CODE_DIR/templates-tarball; $$PATCH_TOOL -s -p1 < $$MESSAGING_PATCH)
|
2020-04-03 14:18:18 +03:00
|
|
|
|
2019-04-04 11:33:38 +03:00
|
|
|
# Build with Sphinx
|
|
|
|
|
cd build
|
introduce new release process (#4513)
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-25 19:01:23 +03:00
|
|
|
sed -i "s,__VERSION__,"{sdk}"," docs/configs/html/conf.py
|
2019-04-04 11:33:38 +03:00
|
|
|
export LC_ALL=en_US.UTF-8
|
|
|
|
|
export LANG=en_US.UTF-8
|
2020-05-11 12:47:54 +03:00
|
|
|
# Sphinx 1.8.3 triggers the following warning:
|
|
|
|
|
#
|
|
|
|
|
# /nix/store/1v39mhhyn48s251przk2fwcvgm71vfqi-python3.7-sphinx-1.8.3/lib/python3.7/site-packages/sphinx/writers/html.py:462: FutureWarning:
|
|
|
|
|
# The iterable returned by Node.traverse()
|
|
|
|
|
# will become an iterator instead of a list in Docutils > 0.16.
|
|
|
|
|
# target_node = image_nodes and image_nodes[0] or node.parent
|
|
|
|
|
#
|
|
|
|
|
# We are using an older Sphinx (1.8.3) with a more recent nixpkgs revision.
|
|
|
|
|
# Unfortunately, an update is not so easy because Sphinx 2.3.1 breaks
|
|
|
|
|
# the PDF documentation due to issues with the FreeSerif font in the
|
|
|
|
|
# fontspec package. So, for now we ignore `FutureWarning`.
|
2019-04-04 11:33:38 +03:00
|
|
|
WARNINGS=$$(../$(location @sphinx_nix//:bin/sphinx-build) -c docs/configs/html docs/source html 2>&1 | \
|
2020-05-11 12:47:54 +03:00
|
|
|
grep -Pi "(?<!future)warning:" || true)
|
2019-04-04 11:33:38 +03:00
|
|
|
|
|
|
|
|
if [ "$$WARNINGS" != "" ]; then
|
|
|
|
|
echo "$$WARNINGS"
|
|
|
|
|
exit 1
|
|
|
|
|
fi
|
|
|
|
|
|
2019-05-21 15:57:09 +03:00
|
|
|
# Copy Javadoc using unzip to avoid having to know the path to the 'jar' binary. Note flag to overwrite
|
|
|
|
|
unzip -o ../$(locations //language-support/java:javadoc) -d html/app-dev/bindings-java/javadocs
|
2020-01-20 18:21:34 +03:00
|
|
|
# Remove JAR metadata
|
|
|
|
|
rm -r html/app-dev/bindings-java/javadocs/META-INF
|
2019-04-04 11:33:38 +03:00
|
|
|
|
2020-03-10 20:46:54 +03:00
|
|
|
# Copy generated documentation for typescript libraries
|
|
|
|
|
mkdir -p html/app-dev/bindings-ts/daml-react
|
|
|
|
|
mkdir -p html/app-dev/bindings-ts/daml-ledger
|
|
|
|
|
mkdir -p html/app-dev/bindings-ts/daml-types
|
|
|
|
|
tar -xzf ../$(location //language-support/ts/daml-react:docs) --strip-components 1 -C html/app-dev/bindings-ts/daml-react/
|
|
|
|
|
tar -xzf ../$(location //language-support/ts/daml-ledger:docs) --strip-components 1 -C html/app-dev/bindings-ts/daml-ledger/
|
|
|
|
|
tar -xzf ../$(location //language-support/ts/daml-types:docs) --strip-components 1 -C html/app-dev/bindings-ts/daml-types/
|
2020-06-03 10:52:17 +03:00
|
|
|
# The generated docs of the typescript libraries are published at two places: The npm
|
|
|
|
|
# registry and on docs.daml.com. The docs at the npm registry contain a link pointing
|
|
|
|
|
# to docs.daml.com. We remove it for the version published at docs.daml.com as it would be
|
|
|
|
|
# pointing to itself.
|
|
|
|
|
sed -i -e 's,^.*\(Comprehensive documentation\|<h2>Documentation</h2>\|0.0.0-SDKVERSION\).*$$,,' html/app-dev/bindings-ts/*/index.html
|
2020-03-10 20:46:54 +03:00
|
|
|
|
2020-05-05 20:35:30 +03:00
|
|
|
# Get the daml cheat sheet
|
|
|
|
|
mkdir -p html/cheat-sheet
|
|
|
|
|
tar -xzf ../$(location @daml-cheat-sheet//:site) --strip-components 1 -C html/cheat-sheet
|
|
|
|
|
|
2019-04-04 11:33:38 +03:00
|
|
|
# Copy in hoogle DB
|
|
|
|
|
mkdir -p html/hoogle_db
|
2020-04-23 18:12:48 +03:00
|
|
|
cp -rL ../$(location //compiler/damlc:daml-base-hoogle.txt) html/hoogle_db/base.txt
|
2019-04-04 11:33:38 +03:00
|
|
|
|
2020-03-31 11:09:52 +03:00
|
|
|
tar c html \
|
|
|
|
|
--owner=0 --group=0 --numeric-owner --mtime=2000-01-01\ 00:00Z --sort=name \
|
|
|
|
|
| gzip -n > ../$(location html-only.tar.gz)
|
introduce new release process (#4513)
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-25 19:01:23 +03:00
|
|
|
""".format(sdk = sdk_version),
|
2020-04-11 22:53:44 +03:00
|
|
|
tools = [
|
|
|
|
|
"@sphinx_nix//:bin/sphinx-build",
|
|
|
|
|
"@patch_dev_env//:patch",
|
|
|
|
|
] + (["@glibc_locales//:locale-archive"] if is_linux else []),
|
2019-06-28 13:55:31 +03:00
|
|
|
) if not is_windows else None
|
2019-04-04 11:33:38 +03:00
|
|
|
|
|
|
|
|
genrule(
|
|
|
|
|
name = "redirects",
|
2019-04-12 14:10:16 +03:00
|
|
|
srcs = [
|
|
|
|
|
"redirects.map",
|
|
|
|
|
"redirect_template.html",
|
|
|
|
|
],
|
2019-04-04 11:33:38 +03:00
|
|
|
outs = ["redirects.tar.gz"],
|
|
|
|
|
cmd = """
|
|
|
|
|
mkdir redirects
|
|
|
|
|
while read l; do
|
|
|
|
|
from=$$(awk -F' -> ' '{print $$1}' <<<$$l)
|
|
|
|
|
to=$$(awk -F' -> ' '{print $$2}' <<<"$$l")
|
|
|
|
|
if [ $$to ]
|
|
|
|
|
then
|
|
|
|
|
mkdir -p redirects/$$(dirname $$from)
|
|
|
|
|
cp -L docs/redirect_template.html redirects/$$from
|
|
|
|
|
sed -i -e "s,__URL__,$${to}," redirects/$$from
|
|
|
|
|
fi
|
|
|
|
|
done <docs/redirects.map
|
2020-03-31 11:09:52 +03:00
|
|
|
tar c redirects \
|
|
|
|
|
--owner=0 --group=0 --numeric-owner --mtime=2000-01-01\ 00:00Z --sort=name \
|
|
|
|
|
| gzip -n > $(location redirects.tar.gz)
|
2019-04-12 14:10:16 +03:00
|
|
|
""",
|
2019-04-04 11:33:38 +03:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
genrule(
|
|
|
|
|
name = "docs",
|
2019-04-12 14:10:16 +03:00
|
|
|
srcs = [
|
|
|
|
|
":docs-no-pdf",
|
|
|
|
|
":pdf-docs",
|
|
|
|
|
":redirects",
|
|
|
|
|
"error.html",
|
|
|
|
|
],
|
2019-04-04 11:33:38 +03:00
|
|
|
outs = ["html.tar.gz"],
|
|
|
|
|
cmd = """
|
2020-01-20 18:21:34 +03:00
|
|
|
VERSION_DATE=$$(cat bazel-out/stable-status.txt | grep STABLE_VERSION_DATE | head -1 | cut -f 2 -d' ')
|
2019-04-04 11:33:38 +03:00
|
|
|
tar -zxf $(location :redirects)
|
|
|
|
|
tar -zxf $(location :docs-no-pdf)
|
|
|
|
|
cp -rn redirects/* html
|
|
|
|
|
cp -L docs/error.html html
|
|
|
|
|
cd html
|
2020-01-20 18:21:34 +03:00
|
|
|
find . -name '*.html' | sort | sed -e 's,^\\./,https://docs.daml.com/,' > sitemap
|
introduce new release process (#4513)
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-25 19:01:23 +03:00
|
|
|
SMHEAD=\"{head}\"
|
|
|
|
|
SMITEM=\"{item}\"
|
|
|
|
|
SMFOOT=\"{foot}\"
|
2019-04-04 11:33:38 +03:00
|
|
|
echo $$SMHEAD > sitemap.xml
|
|
|
|
|
while read item; do
|
2020-01-20 18:21:34 +03:00
|
|
|
echo $$SMITEM | sed -e "s,%DATE%,$${{VERSION_DATE}}," | sed -e "s,%LOC%,$${{item}}," >> sitemap.xml
|
2019-04-04 11:33:38 +03:00
|
|
|
done < sitemap
|
2020-01-20 18:21:34 +03:00
|
|
|
rm sitemap
|
2019-04-04 11:33:38 +03:00
|
|
|
echo $$SMFOOT >> sitemap.xml
|
introduce new release process (#4513)
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-25 19:01:23 +03:00
|
|
|
echo {{ \\"{version}\\" : \\"{version}\\" }} > versions.json
|
2019-04-04 11:33:38 +03:00
|
|
|
cd ..
|
|
|
|
|
cp -L $(location :pdf-docs) html/_downloads
|
2020-01-20 18:21:34 +03:00
|
|
|
# Remove Sphinx build products
|
|
|
|
|
rm -rf .buildinfo .doctrees objects.inv
|
2020-03-31 11:09:52 +03:00
|
|
|
tar c html \
|
|
|
|
|
--owner=0 --group=0 --numeric-owner --mtime=2000-01-01\ 00:00Z --sort=name \
|
|
|
|
|
| gzip -n > $(location html.tar.gz)
|
2019-04-04 11:33:38 +03:00
|
|
|
""".format(
|
introduce new release process (#4513)
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-25 19:01:23 +03:00
|
|
|
head = """<?xml version='1.0' encoding='UTF-8'?><urlset xmlns='http://www.sitemaps.org/schemas/sitemap/0.9' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xsi:schemaLocation='http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd'>""",
|
|
|
|
|
item = """<url><loc>%LOC%</loc><lastmod>%DATE%</lastmod><changefreq>daily</changefreq><priority>0.8</priority></url>""",
|
|
|
|
|
foot = """</urlset>""",
|
|
|
|
|
version = sdk_version,
|
2019-04-12 14:10:16 +03:00
|
|
|
),
|
2020-01-20 18:21:34 +03:00
|
|
|
stamp = 1,
|
2020-02-26 17:52:08 +03:00
|
|
|
tags = ["pdfdocs"],
|
2019-06-28 13:55:31 +03:00
|
|
|
) if not is_windows else None
|
2019-04-04 11:33:38 +03:00
|
|
|
|
2019-05-07 18:41:29 +03:00
|
|
|
filegroup(
|
|
|
|
|
name = "daml-assistant-iou-setup",
|
|
|
|
|
srcs = glob(
|
2020-04-03 18:44:19 +03:00
|
|
|
["source/app-dev/bindings-java/quickstart/template-root/*"],
|
2020-03-09 11:30:18 +03:00
|
|
|
# excluding quickstart-java stuff
|
2019-05-07 18:41:29 +03:00
|
|
|
exclude = [
|
2020-04-03 18:44:19 +03:00
|
|
|
"source/app-dev/bindings-java/quickstart/template-root/src",
|
|
|
|
|
"source/app-dev/bindings-java/quickstart/template-root/pom.xml",
|
2019-05-07 18:41:29 +03:00
|
|
|
],
|
|
|
|
|
exclude_directories = 0,
|
|
|
|
|
),
|
|
|
|
|
visibility = ["//visibility:public"],
|
|
|
|
|
)
|
|
|
|
|
|
2019-04-04 11:33:38 +03:00
|
|
|
genrule(
|
|
|
|
|
name = "quickstart-java",
|
2020-04-03 18:44:19 +03:00
|
|
|
srcs = glob(["source/app-dev/bindings-java/quickstart/template-root/**"]),
|
2019-04-04 11:33:38 +03:00
|
|
|
outs = ["quickstart-java.tar.gz"],
|
|
|
|
|
cmd = """
|
|
|
|
|
mkdir -p quickstart-java
|
2020-04-03 18:44:19 +03:00
|
|
|
cp -rL docs/source/app-dev/bindings-java/quickstart/template-root/* quickstart-java/
|
introduce new release process (#4513)
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-25 19:01:23 +03:00
|
|
|
sed -i "s/__VERSION__/{mvn}/" quickstart-java/pom.xml
|
2020-03-31 11:09:52 +03:00
|
|
|
tar c quickstart-java \
|
|
|
|
|
--owner=0 --group=0 --numeric-owner --mtime=2000-01-01\ 00:00Z --sort=name \
|
|
|
|
|
| gzip -n > $@
|
introduce new release process (#4513)
Context
=======
After multiple discussions about our current release schedule and
process, we've come to the conclusion that we need to be able to make a
distinction between technical snapshots and marketing releases. In other
words, we need to be able to create a bundle for early adopters to test
without making it an officially-supported version, and without
necessarily implying everyone should go through the trouble of
upgrading. The underlying goal is to have less frequent but more stable
"official" releases.
This PR is a proposal for a new release process designed under the
following constraints:
- Reuse as much as possible of the existing infrastructure, to minimize
effort but also chances of disruptions.
- Have the ability to create "snapshot"/"nightly"/... releases that are
not meant for general public consumption, but can still be used by savvy
users without jumping through too many extra hoops (ideally just
swapping in a slightly-weirder version string).
- Have the ability to promote an existing snapshot release to "official"
release status, with as few changes as possible in-between, so we can be
confident that the official release is what we tested as a prerelease.
- Have as much of the release pipeline shared between the two types of
releases, to avoid discovering non-transient problems while trying to
promote a snapshot to an official release.
- Triggerring a release should still be done through a PR, so we can
keep the same approval process for SOC2 auditability.
The gist of this proposal is to replace the current `VERSION` file with
a `LATEST` file, which would have the following format:
```
ef5d32b7438e481de0235c5538aedab419682388 0.13.53-alpha.20200214.3025.ef5d32b7
```
This file would be maintained with a script to reduce manual labor in
producing the version string. Other than that, the process will be
largely the same, with releases triggered by changes to this `LATEST`
and the release notes files.
Version numbers
===============
Because one of the goals is to reduce the velocity of our published
version numbers, we need a different version scheme for our snapshot
releases. Fortunately, most version schemes have some support for that;
unfortunately, the SDK sits at the intersection of three different
version schemes that have made incompatible choices. Without going into
too much detail:
- Semantic versioning (which we chose as the version format for the SDK
version number) allows for "prerelease" version numbers as well as
"metadata"; an example of a complete version string would be
`1.2.3-nightly.201+server12.43`. The "main" part of the version string
always has to have 3 numbers separated by dots; the "prerelease"
(after the `-` but before the `+`) and the "metadata" (after the `+`)
parts are optional and, if present, must consist of one or more segments
separated by dots, where a segment can be either a number or an
alphanumeric string. In terms of ordering, metadata is irrelevant and
any version with a prerelease string is before the corresponding "main"
version string alone. Amongst prereleases, segments are compared in
order with purely numeric ones compared as numbers and mixed ones
compared lexicographically. So 1.2.3 is more recent than 1.2.3-1,
which is itself less recent than 1.2.3-2.
- Maven version strings are any number of segments separated by a `.`, a
`-`, or a transition between a number and a letter. Version strings
are compared element-wise, with numeric segments being compared as
numbers. Alphabetic segments are treated specially if they happen to be
one of a handful of magic words (such as "alpha", "beta" or "snapshot"
for example) which count as "qualifiers"; a version string with a
qualifier is "before" its prefix (`1.2.3` is before `1.2.3-alpha.3`,
which is the same as `1.2.3-alpha3` or `1.2.3-alpha-3`), and there is a
special ordering amongst qualifiers. Other alphabetic segments are
compared alphabetically and count as being "after" their prefix
(`1.2.3-really-final-this-time` counts as being released after `1.2.3`).
- GHC package numbers are comprised of any number of numeric segments
separated by `.`, plus an optional (though deprecated) alphanumeric
"version tag" separated by a `-`. I could not find any official
documentation on ordering for the version tag; numeric segments are
compared as numbers.
- npm uses semantic versioning so that is covered already.
After much more investigation than I'd care to admit, I have come up
with the following compromise as the least-bad solution. First,
obviously, the version string for stable/marketing versions is going to
be "standard" semver, i.e. major.minor.patch, all numbers, which works,
and sorts as expected, for all three schemes. For snapshot releases, we
shall use the following (semver) format:
```
0.13.53-alpha.20200214.3025.ef5d32b7
```
where the components are, respectively:
- `0.13.53`: the expected version string of the next "stable" release.
- `alpha`: a marker that hopefully scares people enough.
- `20200214`: the date of the release commit, which _MUST_ be on
master.
- `3025`: the number of commits in master up to the release commit
(included). Because we have a linear, append-only master branch, this
uniquely identifies the commit.
- `ef5d32b7ù : the first 8 characters of the release commit sha. This is
not strictly speaking necessary, but makes it a lot more convenient to
identify the commit.
The main downsides of this format are:
1. It is not a valid format for GHC packages. We do not publish GHC
packages from the SDK (so far we have instead opted to release our
Haskell code as separate packages entirely), so this should not be an
issue. However, our SDK version currently leaks to `ghc-pkg` as the
version string for the stdlib (and prim) packages. This PR addresses
that by tweaking the compiler to remove the offending bits, so `ghc-pkg`
would see the above version number as `0.13.53.20200214.3025`, which
should be enough to uniquely identify it. Note that, as far as I could
find out, this number would never be exposed to users.
2. It is rather long, which I think is good from a human perspective as
it makes it more scary. However, I have been told that this may be
long enough to cause issues on Windows by pushing us past the max path
size limitation of that "OS". I suggest we try it and see what
happens.
The upsides are:
- It clearly indicates it is an unstable release (`alpha`).
- It clearly indicates how old it is, by including the date.
- To humans, it is immediately obvious which version is "later" even if
they have the same date, allowing us to release same-day patches if
needed. (Note: that is, commits that were made on the same day; the
release date itself is irrelevant here.)
- It contains the git sha so the commit built for that release is
immediately obvious.
- It sorts correctly under all schemes (modulo the modification for
GHC).
Alternatives I considered:
- Pander to GHC: 0.13.53-alpha-20200214-3025-ef5d32b7. This format would
be accepted by all schemes, but will not sort as expected under semantic
versioning (though Maven will be fine). I have no idea how it will sort
under GHC.
- Not having any non-numeric component, e.g. `0.13.53.20200214.3025`.
This is not valid semantic versioning and is therefore rejected by
npm.
- Not having detailed info: just go with `0.13.53-snapshot`. This is
what is generally done in the Java world, but we then lose track of what
version is actually in use and I'm concerned about bug reports. This
would also not let us publish to the main Maven repo (at least not more
than once), as artifacts there are supposed to be immutable.
- No having a qualifier: `0.13.53-3025` would be acceptable to all three
version formats. However, it would not clearly indicate to humans that
it is not meant as a stable version, and would sort differently under
semantic versioning (which counts it as a prerelease, i.e. before
`0.13.53`) than under maven (which counts it as a patch, so after
`0.13.53`).
- Just counting releases: `0.13.53-alpha.1`, where we just count the
number of prereleases in-between `0.13.52` and the next. This is
currently the fallback plan if Windows path length causes issues. It
would be less convenient to map releases to commits, but it could still
be done via querying the history of the `LATEST` file.
Release notes
=============
> Note: We have decided not to have release notes for snapshot releases.
Release notes are a bit tricky. Because we want the ability to make
snapshot releases, then later on promote them to stable releases, it
follows that we want to build commits from the past. However, if we
decide post-hoc that a commit is actually a good candidate for a
release, there is no way that commit can have the appropriate release
notes: it cannot know what version number it's getting, and, moreover,
we now track changes in commit messages. And I do not think anyone wants
to go back to the release notes file being a merge bottleneck.
But release notes need to be published to the releases blog upon
releasing a stable version, and the docs website needs to be updated and
include them.
The only sensible solution here is to pick up the release notes as of
the commit that triggers the release. As the docs cron runs
asynchronously, this means walking down the git history to find the
relevant commit.
> Note: We could probably do away with the asynchronicity at this point.
> It was originally included to cover for the possibility of a release
> failing. If we are releasing commits from the past after they have been
> tested, this should not be an issue anymore. If the docs generation were
> part of the synchronous release step, it would have direct access to the
> correct release notes without having to walk down the git history.
>
> However, I think it is more prudent to keep this change as a future step,
> after we're confident the new release scheme does indeed produce much more
> reliable "stable" releases.
New release process
===================
Just like releases are currently controlled mostly by detecting
changes to the `VERSION` file, the new process will be controlled by
detecting changes to the `LATEST` file. The format of that file will
include both the version string and the corresponding SHA.
Upon detecting a change to the `LATEST` file, CI will run the entire
release process, just like it does now with the VERSION file. The main
differences are:
1. Before running the release step, CI will checkout the commit
specified in the LATEST file. This requires separating the release
step from the build step, which in my opinion is cleaner anyway.
2. The `//:VERSION` Bazel target is replaced by a repository rule
that gets the version to build from an environment variable, with a
default of `0.0.0` to remain consistent with the current `daml-head`
behaviour.
Some of the manual steps will need to be skipped for a snapshot release.
See amended `release/RELEASE.md` in this commit for details.
The main caveat of this approach is that the official release will be a
different binary from the corresponding snapshot. It will have been
built from the same source, but with a different version string. This is
somewhat mitigated by Bazel caching, meaning any build step that does
not depend on the version string should use the cache and produce
identical results. I do not think this can be avoided when our artifact
includes its own version number.
I must note, though, that while going through the changes required after
removing the `VERSION` file, I have been quite surprised at the sheer number of
things that actually depend on the SDK version number. I believe we should
look into reducing that over time.
CHANGELOG_BEGIN
CHANGELOG_END
2020-02-25 19:01:23 +03:00
|
|
|
""".format(mvn = mvn_version),
|
2019-04-04 11:33:38 +03:00
|
|
|
visibility = ["//visibility:public"],
|
|
|
|
|
)
|
|
|
|
|
|
2019-04-08 17:26:34 +03:00
|
|
|
load("//language-support/java/codegen:codegen.bzl", "dar_to_java")
|
2019-04-04 11:33:38 +03:00
|
|
|
|
2020-03-09 11:30:18 +03:00
|
|
|
genrule(
|
2019-04-12 14:10:16 +03:00
|
|
|
name = "quickstart-model",
|
2020-03-09 11:30:18 +03:00
|
|
|
srcs = [
|
2020-04-03 18:44:19 +03:00
|
|
|
"//docs:source/app-dev/bindings-java/quickstart/template-root/daml/Main.daml",
|
|
|
|
|
"//docs:source/app-dev/bindings-java/quickstart/template-root/daml/Iou.daml",
|
|
|
|
|
"//docs:source/app-dev/bindings-java/quickstart/template-root/daml/IouTrade.daml",
|
|
|
|
|
"//docs:source/app-dev/bindings-java/quickstart/template-root/daml/Setup.daml",
|
2020-03-09 11:30:18 +03:00
|
|
|
"//daml-script/daml:daml-script.dar",
|
HTTP JSON API first version (#1994)
* Cleanup
* WIP
* first integration test + fixture
* minor cleanup
* Implementing ContractService.lookup
* Reverting back to endpoints.all (all2 did not work)
* Cleanup
* replace ApiValue ADT with aliases to daml-lf/transaction Value ADT
* porting rest of navigator to LF Value ADT
* Command Service WIP
* CommandService WIP
* porting more of navigator to LF Value ADT
* last error, not first
* rename ApiValueImplicits file
* special conversion features for ImmArray and FrontStack
- just .to[ImmArray] or .to[FrontStack] any random collection
* finish porting most of navigator main code
* use numeric indices for record field name fallback when pretty-printing
* tuples are not serializable
* use numeric indices for label fallback in JSON verbose encoding
* make traverseEitherStrictly more likely to preserve the seq's class
* to shortcut for ImmArraySeq .to[ImmArraySeq]
* compiling, passing navigator backend tests
* test traverseEitherStrictly more, er, strictly
* pass scalacopts through to scaladoc
* deal with unused warning
* remove unneeded function
* simpler error reporting, more private functions in ApiCodecCompressed
* move slowApply to FrontStack, test it so it actually works
* remove unneeded toStrings; better error from impossible ValueTuple case
* scalafmt FrontStackSpec
* support alternative, label-free record JSON encoding
* Adding domain.CreateCommand + corresponding json formats and dummy json format for lav1.value.Record
* CommandService.create should be done... need to test it
* TODO added
* Cleanup
* move ApiCodecCompressed, ApiValueImplicits, and some aliases to new lf-value-json package
* Using tagged TemplateId type instead of Identifier + exercise command WIP
* adapt navigator to moved pieces
* start defining scalacheck extension to ApiCodecCompressedSpec
* CommandService.exercise + introducing CommandMeta
* Adding command endpoints, can't test them yet, need lf value json formats
* fuse some list operations
- suggested by @stefanobaghino-da; thanks
* blue error message
* Minor fixes after merging librify-navigator-json-compressed, #2136
* experiment with an inductive case in TypedValueGenerators
* finish a List case for TypedValueGenerators; it's revealing
* Introducing API value to LF value converter,
CommandsValidator takes IdentifierResolverLike instead of IdentifierResolver
* cleanup
* remove accidentally readded duplicate aliases
* start tying knots in TypedValueGenerators
* verbatim copy ApiCodecCompressedSpec to lf-value-json
* shift some tests from navigator to lf-value-json
* test Optional and Map for ApiCodecCompressed
* heavier random testing of ApiCodecCompressed
* remove unused dependencies from lf-value-json
* adding value json writer
* cleanup
* Revert "cleanup"
This reverts commit 2e4d153f
* fixing the build
* cleanup
* cleaning up imports
* JsValue to API value is done, needs a test
* cleanup
* use scalac -Ypartial-unification in http-json
* simplify some Traverse instances
* factor CreateCommand and ExerciseCommand traverse instances
* Command create integration test WIP
* Command create integration test WIP, got rid of the JsonReader and JsonWriter for the values, converting values explicitly
* Extracting DomainJsonDecoder and DomainJsonEncoder
* LfV refactoring
* Create command serialize/deserialize test works
* cleanup
* resolving conflicts
* More json encode/decode tests
* logging
* command/create passes integration test now
* Adding readme
* grammar
* TODO added
* GetActiveContractsResponse encoding
* ideintifier conversion renaming
* PackageService resolveTemplateId returns domain.TemplateId now
* Resolving LF Identifier instead of Template ID, this should also work for Exercise command decoding
* cleaning up a bit
* daml-lf: show type in TypedValueGenerators-driven errors
* exercise command json encoding/decoding works
* command/exercise IOU_Transfer integration test passes now
* avoid filter for Gens; makes many contract ID gens not fail
* test ApiCodecCompressed against 100 random types, 20 random values each
* Updating README instructions
* improving error handling, failed futures, get logged and reported to the user now as 500
* [ROUTING DSL] Removing routing DSL, it did not work
* getting rid of HttpEntity.Strict match + cleanup
* fixing the merge conflict
* updating README
* use Show.shows instead of new Show
* List(_) isn't checked, but Seq(_) is slightly safer
* improving test assertions
* Adding /contracts/lookup implementation
* http-json: use ImmArraySeq instead of List; use toRightDisjuction
* http-json: .toList.toSet is shorter than fold
* http-json: replace .leftMap.map with .bimap
* http-json: use subst instead of reimplementing JsonFormat
* http-json: remove unused ExceptionHandler
* http-json: safer == comparison
* Adding two test cases for expected errors
* Adding BazelRunfiles.rlocation magic that supposed to handle windows path for bazel dependencies
* http-json: import, not extend
2019-07-29 23:49:57 +03:00
|
|
|
],
|
2020-03-09 11:30:18 +03:00
|
|
|
outs = ["quickstart-model.dar"],
|
|
|
|
|
cmd = """
|
|
|
|
|
set -eou pipefail
|
|
|
|
|
TMP_DIR=$$(mktemp -d)
|
|
|
|
|
mkdir -p $$TMP_DIR/daml
|
2020-04-03 18:44:19 +03:00
|
|
|
cp -R -L $(location //docs:source/app-dev/bindings-java/quickstart/template-root/daml/Main.daml) $$TMP_DIR/daml/
|
|
|
|
|
cp -R -L $(location //docs:source/app-dev/bindings-java/quickstart/template-root/daml/Iou.daml) $$TMP_DIR/daml/
|
|
|
|
|
cp -R -L $(location //docs:source/app-dev/bindings-java/quickstart/template-root/daml/IouTrade.daml) $$TMP_DIR/daml/
|
|
|
|
|
cp -R -L $(location //docs:source/app-dev/bindings-java/quickstart/template-root/daml/Setup.daml) $$TMP_DIR/daml/
|
2020-03-09 11:30:18 +03:00
|
|
|
cp -L $(location //daml-script/daml:daml-script.dar) $$TMP_DIR/
|
|
|
|
|
cat << EOF > $$TMP_DIR/daml.yaml
|
|
|
|
|
sdk-version: {sdk}
|
|
|
|
|
name: quickstart-model
|
|
|
|
|
source: daml
|
|
|
|
|
version: 0.0.1
|
|
|
|
|
dependencies:
|
|
|
|
|
- daml-stdlib
|
|
|
|
|
- daml-prim
|
|
|
|
|
- daml-script.dar
|
|
|
|
|
build-options: ["--ghc-option", "-Werror"]
|
|
|
|
|
EOF
|
|
|
|
|
$(location //compiler/damlc) build --project-root=$$TMP_DIR -o $$PWD/$(location quickstart-model.dar)
|
|
|
|
|
rm -rf $$TMP_DIR
|
|
|
|
|
""".format(sdk = sdk_version),
|
|
|
|
|
tools = ["//compiler/damlc"],
|
|
|
|
|
visibility = ["//visibility:public"],
|
2019-04-08 17:26:34 +03:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
dar_to_java(
|
2019-04-04 11:33:38 +03:00
|
|
|
name = "quickstart-model",
|
2019-04-08 17:26:34 +03:00
|
|
|
src = "quickstart-model.dar",
|
2020-04-05 20:49:57 +03:00
|
|
|
package_prefix = "com.daml.quickstart.model",
|
2019-04-04 11:33:38 +03:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
java_binary(
|
|
|
|
|
name = "quickstart-java-lib",
|
2020-04-03 18:44:19 +03:00
|
|
|
srcs = glob(["source/app-dev/bindings-java/quickstart/template-root/src/main/java/**/*.java"]) + [":quickstart-model-srcjar"],
|
2020-04-05 20:49:57 +03:00
|
|
|
main_class = "com.daml.quickstart.iou.IouMain",
|
2019-04-04 11:33:38 +03:00
|
|
|
deps = [
|
2019-10-15 12:36:11 +03:00
|
|
|
"//daml-lf/archive:daml_lf_dev_archive_java_proto",
|
2019-04-04 11:33:38 +03:00
|
|
|
"//language-support/java/bindings:bindings-java",
|
2019-04-12 14:10:16 +03:00
|
|
|
"//language-support/java/bindings-rxjava",
|
2019-10-28 15:53:14 +03:00
|
|
|
"@maven//:ch_qos_logback_logback_classic",
|
|
|
|
|
"@maven//:com_google_code_gson_gson",
|
|
|
|
|
"@maven//:com_google_guava_guava",
|
|
|
|
|
"@maven//:com_google_protobuf_protobuf_java",
|
|
|
|
|
"@maven//:com_sparkjava_spark_core",
|
|
|
|
|
"@maven//:io_reactivex_rxjava2_rxjava",
|
|
|
|
|
"@maven//:org_slf4j_slf4j_api",
|
2019-04-12 14:10:16 +03:00
|
|
|
],
|
2019-04-04 11:33:38 +03:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
daml_test(
|
2019-06-07 22:18:21 +03:00
|
|
|
name = "ledger-api-daml-test",
|
|
|
|
|
srcs = glob(["source/app-dev/code-snippets/**/*.daml"]),
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
daml_test(
|
|
|
|
|
name = "bindings-java-daml-test",
|
|
|
|
|
srcs = glob(["source/app-dev/bindings-java/code-snippets/**/*.daml"]),
|
2019-04-04 11:33:38 +03:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
daml_test(
|
|
|
|
|
name = "patterns-daml-test",
|
2019-04-12 14:10:16 +03:00
|
|
|
srcs = glob(["source/daml/patterns/daml/**/*.daml"]),
|
2019-04-04 11:33:38 +03:00
|
|
|
)
|
|
|
|
|
|
2020-03-30 15:35:22 +03:00
|
|
|
pkg_tar(
|
|
|
|
|
name = "daml-patterns",
|
|
|
|
|
srcs = glob(["source/daml/patterns/daml/**/*.daml"]) + ["source/daml/patterns/daml.yaml.template"],
|
|
|
|
|
strip_prefix = "/docs/source/daml/patterns",
|
|
|
|
|
visibility = ["//visibility:public"],
|
|
|
|
|
)
|
|
|
|
|
|
2019-04-04 11:33:38 +03:00
|
|
|
daml_test(
|
|
|
|
|
name = "daml-studio-daml-test",
|
2019-04-12 14:10:16 +03:00
|
|
|
srcs = glob(["source/daml/daml-studio/daml/**/*.daml"]),
|
2019-04-04 11:33:38 +03:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
daml_test(
|
|
|
|
|
name = "daml-ref-daml-test",
|
2019-04-04 20:58:56 +03:00
|
|
|
timeout = "long",
|
2019-04-12 14:10:16 +03:00
|
|
|
srcs = glob(["source/daml/code-snippets/**/*.daml"]),
|
2019-04-04 11:33:38 +03:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
daml_test(
|
|
|
|
|
name = "introduction-daml-test",
|
2019-04-12 14:10:16 +03:00
|
|
|
srcs = glob(["source/getting-started/introduction/code/**/*.daml"]),
|
2019-04-04 11:33:38 +03:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
daml_test(
|
|
|
|
|
name = "quickstart-daml-test",
|
2020-03-09 11:30:18 +03:00
|
|
|
srcs = glob(
|
2020-04-03 18:44:19 +03:00
|
|
|
include = ["source/app-dev/bindings-java/quickstart/template-root/daml/**/*.daml"],
|
2020-03-09 11:30:18 +03:00
|
|
|
# excluding Setup.daml, because it doesn't contain any scenarios,
|
|
|
|
|
# and we cannot conveniently add daml-script as a dependency for daml_test
|
2020-04-03 18:44:19 +03:00
|
|
|
exclude = ["source/app-dev/bindings-java/quickstart/template-root/daml/Setup.daml"],
|
2020-03-09 11:30:18 +03:00
|
|
|
),
|
2019-04-04 11:33:38 +03:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
daml_test(
|
|
|
|
|
name = "ledger-model-daml-test",
|
2019-04-12 14:10:16 +03:00
|
|
|
srcs = glob(["source/concepts/ledger-model/daml/**/*.daml"]),
|
2019-04-04 11:33:38 +03:00
|
|
|
)
|
2019-04-11 15:36:14 +03:00
|
|
|
|
|
|
|
|
daml_test(
|
|
|
|
|
name = "java-bindings-docs-daml-test",
|
2019-04-12 14:10:16 +03:00
|
|
|
srcs = glob(["source/app-dev/bindings-java/daml/**/*.daml"]),
|
2019-04-11 15:36:14 +03:00
|
|
|
)
|
2019-06-18 12:56:06 +03:00
|
|
|
|
|
|
|
|
daml_test(
|
|
|
|
|
name = "daml-intro-daml-test",
|
|
|
|
|
srcs = glob(["source/daml/intro/daml/**/*.daml"]),
|
|
|
|
|
)
|
|
|
|
|
|
2020-03-10 17:23:23 +03:00
|
|
|
daml_build_test(
|
|
|
|
|
name = "daml-upgrade-example-v1",
|
2020-03-10 19:24:17 +03:00
|
|
|
project_dir = "source/upgrade/example/coin-1.0.0",
|
2020-03-10 17:23:23 +03:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
daml_build_test(
|
|
|
|
|
name = "daml-upgrade-example-v2",
|
2020-03-10 19:24:17 +03:00
|
|
|
project_dir = "source/upgrade/example/coin-2.0.0",
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
daml_build_test(
|
|
|
|
|
name = "daml-upgrade-example-upgrade",
|
2020-03-10 17:23:23 +03:00
|
|
|
dar_dict = {
|
2020-03-10 19:24:17 +03:00
|
|
|
":daml-upgrade-example-v1": "path/to/coin-1.0.0.dar",
|
|
|
|
|
":daml-upgrade-example-v2": "path/to/coin-2.0.0.dar",
|
2020-03-10 17:23:23 +03:00
|
|
|
},
|
2020-03-10 19:24:17 +03:00
|
|
|
project_dir = "source/upgrade/example/coin-upgrade",
|
2020-03-02 11:54:39 +03:00
|
|
|
)
|
|
|
|
|
|
2020-04-07 15:36:40 +03:00
|
|
|
daml_build_test(
|
|
|
|
|
name = "daml-upgrade-example-upgrade-script",
|
|
|
|
|
dar_dict = {
|
|
|
|
|
":daml-upgrade-example-v1": "path/to/coin-1.0.0.dar",
|
|
|
|
|
":daml-upgrade-example-v2": "path/to/coin-2.0.0.dar",
|
|
|
|
|
":daml-upgrade-example-upgrade": "path/to/coin-upgrade-1.0.0.dar",
|
|
|
|
|
"//daml-script/daml:daml-script.dar": "path/to/daml-script.dar",
|
|
|
|
|
},
|
|
|
|
|
project_dir = "source/upgrade/example/coin-initiate-upgrade",
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
daml_build_test(
|
|
|
|
|
name = "daml-upgrade-example-upgrade-trigger",
|
|
|
|
|
dar_dict = {
|
|
|
|
|
":daml-upgrade-example-v1": "path/to/coin-1.0.0.dar",
|
|
|
|
|
":daml-upgrade-example-v2": "path/to/coin-2.0.0.dar",
|
|
|
|
|
":daml-upgrade-example-upgrade": "path/to/coin-upgrade-1.0.0.dar",
|
|
|
|
|
"//triggers/daml:daml-trigger.dar": "path/to/daml-trigger.dar",
|
|
|
|
|
},
|
|
|
|
|
project_dir = "source/upgrade/example/coin-upgrade-trigger",
|
|
|
|
|
)
|
|
|
|
|
|
2019-06-18 12:56:06 +03:00
|
|
|
filegroup(
|
|
|
|
|
name = "daml-intro-1",
|
|
|
|
|
srcs = glob(
|
|
|
|
|
["source/daml/intro/daml/1_Token/**/*"],
|
2020-03-09 11:30:18 +03:00
|
|
|
# excluding quickstart-java stuff
|
2019-06-18 12:56:06 +03:00
|
|
|
exclude = [
|
2020-04-03 18:44:19 +03:00
|
|
|
"source/app-dev/bindings-java/quickstart/template-root/src",
|
|
|
|
|
"source/app-dev/bindings-java/quickstart/template-root/pom.xml",
|
2019-06-18 12:56:06 +03:00
|
|
|
],
|
|
|
|
|
exclude_directories = 0,
|
|
|
|
|
),
|
|
|
|
|
visibility = ["//visibility:public"],
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
pkg_tar(
|
|
|
|
|
name = "daml-intro-templates",
|
|
|
|
|
srcs = glob(["source/daml/intro/daml/**"]),
|
|
|
|
|
strip_prefix = "source/daml/intro/daml",
|
|
|
|
|
visibility = ["//visibility:public"],
|
|
|
|
|
)
|
2019-10-15 17:50:43 +03:00
|
|
|
|
|
|
|
|
pkg_tar(
|
|
|
|
|
name = "copy-trigger-template",
|
|
|
|
|
srcs = glob(
|
|
|
|
|
["source/triggers/template-root/**"],
|
|
|
|
|
exclude = ["**/*~"],
|
|
|
|
|
),
|
|
|
|
|
strip_prefix = "source/triggers/template-root",
|
|
|
|
|
visibility = ["//visibility:public"],
|
|
|
|
|
)
|
2019-10-17 15:52:15 +03:00
|
|
|
|
2019-11-19 20:19:57 +03:00
|
|
|
pkg_tar(
|
|
|
|
|
name = "script-example-template",
|
|
|
|
|
srcs = glob(
|
|
|
|
|
["source/daml-script/template-root/**"],
|
|
|
|
|
exclude = ["**/*~"],
|
|
|
|
|
),
|
|
|
|
|
strip_prefix = "source/daml-script/template-root",
|
|
|
|
|
visibility = ["//visibility:public"],
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
exports_files([
|
|
|
|
|
"source/triggers/template-root/src/CopyTrigger.daml",
|
|
|
|
|
"source/daml-script/template-root/src/ScriptExample.daml",
|
|
|
|
|
])
|
