With the new 2.0.0 release process, we're not releasing from the main CI
build anymore, so the compat update PRs are not getting created.
This restores that by making those PR creations part of the daily CI
build.
CHANGELOG_BEGIN
CHANGELOG_END
This should hopefully get CI working again.
There are two changes in here:
1. We can no longer change our patched Bazel. I didn’t switch away
from the current patched version for now (we upload it to gcp bucket
so it still works fine even if we cannot build it) but if we upgrade,
we need to go to an unpatched version for now.
2. We need to get `az` from dev-env. I tested the self service compat
job stuff and it works fine with this but there is a chance other
parts don’t.
changelog_begin
changelog_end
Because there's no reason not to. The only obstacle to automating the
normal release process is that we need an explicit manual validation
step for our audit log when creating a release, but split-releases are
created in the Assemblty repo, so we can have the audit log over there.
The diff is going to be very messy because there's a lot of stuff moving
around. The only thing that is not just moving a job to a separate file
is the `ci/daily-snapshot.yml` file, which is 100% new and is meant as
a new Azure Pipelines entrypoint (which I will create after this gets
approved). I have made a reasonable effort to create individual commits
that simplify reviewing, but I expect it's still going to be kind of a
mess. I'm open to opening separate PRs to ~bump my stats~ move one job
at a time if that makes reviewing (and testing) easier.
CHANGELOG_BEGIN
CHANGELOG_END
We use tags to track releases; in particular, we want tags to correspond
to GitHub releases. In the split release process, the `assembly` repo is
responsible for creating GitHub releases, so it would make sense that it
is also the one responsible for creating tags.
I would also like to take this opportunity to manually remove all the
tags that have been created by the split release process so far and that
don't correspond to any release (which is all of them as we haven't done
a full split release yet), since they have already caused some
confusion.
CHANGELOG_BEGIN
CHANGELOG_END
Apologies for doing two things in one PR but they seemed somewhat
entangled.
For artifactory we follow the approach we use for Maven artifacts and
publish there directly.
For the SDK EE tarball, we just throw it in the split-release
directory and then the assembly repo can pick it up.
changelog_begin
changelog_end
This is a private bucket which allows us to publish EE artifacts.
We could separate public from private artifacts but keeping everything
in a single location is easier to manage.
changelog_begin
changelog_end
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.
I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.
CHANGELOG_BEGIN
CHANGELOG_END
* Split release process (PoC)
This PR adds the necessary infrastructure for the new split release
process. Releases are still triggered via the LATEST file but you can
choose between the old and the new release process by adding
SPLIT-RELEASE at the end of the line.
As a first step this publishes all artifacts we currently publish to
Github releases to our GCP bucket.
changelog_begin
changelog_end
* drop report-start
changelog_begin
changelog_end
* fix gcp bucket
changelog_begin
changelog_end
* review feedback
changelog_begin
changelog_end
* Update azure-pipelines.yml
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* SPLIT-RELEASE ->SPLIT_RELEASE
changelog_begin
changelog_end
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* Drop snapshots from compat tests
We don’t really give strict compatibility guarantees on snapshots and
we actively break them (e.g. you cannot migrate across snapshots). So
this PR stops including snapshots in the compat tests which should
both speed them up a bit and stop irrelevant failures.
changelog_begin
changelog_end
* hlint
changelog_begin
changelog_end
The "tell slack about failures" step is conditional on the branch name
being main; for whatever reason if we don't supply a branch name Azure
sets it to `main`, regardless of the fact that the commit is not in that
branch at all.
CHANGELOG_BEGIN
CHANGELOG_END
It should now be possible for anyone to trigger the daily compat build
by adding a `run-full-compat: true` trailer to their commit message.
CHANGELOG_BEGIN
CHANGELOG_END
run-full-compat: true
Reminds people adding new users to the rotation that they should also
be added to DACH-NY/daml-language-ad-hoc to have a box in the cloud
for Windows testing.
Adds the possibility of adding comments at the beginning of the rotation file.
changelog_begin
changelog_end
This PR drops two things:
1. The check that the benchmark hasn’t been modified. This hasn’t ever
been useful and it keeps being annoying.
2. It stops the comparison against the old version and instead just
benchmarks the current version. We really only care about the day to
day changes. Comparing against an arbitrary year old version has lost
all meaning at this point.
changelog_begin
changelog_end
Now that we skip the scala_2_12 job this dep results in us never
running the release job which is clearly not intentional.
changelog_begin
changelog_end
* maybe fix deleted release tags
Hard to know for sure until the next broken release.
CHANGELOG_BEGIN
CHANGELOG_END
* Update azure-pipelines.yml
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Ubuntu 16.04 nodes are about to go away and there dosen’t seem to be a
compelling reason to use the hosted nodes here anyway.
This is obviously hard/impossible to test without a release but I did
do some basic sanity checks: The release build seems to rely on the
following tools (+ things like rm which are clearly there)
1. git
2. gpg
3. sha256sum
4. gcloud
5. curl
All of those are installed on our own nodes. If this doesn’t work we
could also run (parts of) this in dev-env but for now this should do
the job.
changelog_begin
changelog_end
.
changelog_begin
changelog_end
.
changelog_begin
changelog_end
Three issues here:
1. The release job runs on an Azure-hosted agent, so it doesn't have the
`reset_caches.sh` script (and doesn't need it).
2. The `bash-lib` step should not run if the current job has already
failed.
3. The `skip-github` jobs should also not run if the job has failed.
CHANGELOG_BEGIN
CHANGELOG_END
The revamped release process relies on publish.sh coming from the
release tag. However without the checkout that doesn’t work. This
resulted in us trying to push the EE tarball for a branch that does
not have it yet.
changelog_begin
changelog_end
* Move artifact publishing out of yaml files
The current publishing process pretty much hardcodes the set of
artifacts we publish in the yaml config. This is a problem because we
always release from `main` so the yaml files are always
identical. However, we will add new artifacts over time and this
starts falling apart. This PR changes this such that the process
described in the yaml files is very generic and just uploads and
downloads everything in a directory whereas the details are handled in
bash scripts that will come from the respective release branch and are
therefore version-dependent.
As usual for these type of changes, I don’t have a great way to test
this. I did do some due diligence to test that at least the artifacts
are published correctly and I can download them but I can’t test the
actual publishing.
changelog_begin
changelog_end
* Update ci/copy-unix-release-artifacts.sh
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* Update ci/copy-windows-release-artifacts.sh
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* Update ci/publish-artifactory.sh
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
The people who care about these alerts monitor the channel closely
enough anyway, and having frequent automated @here bells ringing makes
it harder for individuals to highlight important messages.
CHANGELOG_BEGIN
CHANGELOG_END
My goal here is to investigate the new warning Azure has been showing
for the past few days:
> ##[warning]%25 detected in ##vso command. In March 2021, the agent command parser will be updated to unescape this to %. To opt out of this behavior, set a job level variable DECODE_PERCENTS to false. Setting to true will force this behavior immediately. More information can be found at https://github.com/microsoft/azure-pipelines-agent/blob/master/docs/design/percentEncoding.md
As far as I'm aware we are not deliberately passing in any `%25` in any
of our `vso` commands, so I was a bit surprised by this.
CHANGELOG_BEGIN
CHANGELOG_END
CHANGELOG_BEGIN
- Our Linux binaries are now built on Ubuntu 20.04 instead of 16.04. We
do not expect any user-level impact, but please reach out if you
do notice any issue that might be caused by this.
CHANGELOG_END
* include oauth2 logback config in release tarball
overlooked in https://github.com/digital-asset/daml/pull/8611
* Release trigger-service and oauth2-middleware JARs
changelog_begin
changelog_end
* drop from artifacts.yaml
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
