I _believe_ this is what has been tripping up the check-releses job for
the past few days: while updating the signing key, I did not update the
key fingerprint to match, and therefore trying to check any signature
fails.
The failure mode of the existing script is still pretty bad, regadless:
looping on the same release until the hard drive is full is definitely
not expected. Our retry policies are supposed to have time bounds, and
failure should result in cleaning up the workdir.
run-full-compat: true
There's a small, subtle bug that made its way into #14287: apparently
`/dev/stdout` is not defined on CI.
On the bright side, we don't actually need the `out` parameter, so
removing it seems like the easiest fix here.
CHANGELOG_BEGIN
CHANGELOG_END
* Merge ci/cron/wednesday.yml open_pr function into ci/bash-lib.yml
* remove unused function reset from ci/cron/wednesday.yml
* Request review from current release person for rotation PR
changelog_begin
changelog_end
The check_releases job has been a major player in the flakiness of the
daily test lately, simply by _timing out_ despite its 6h limit.
There are smarter, more "permanent" fixes we could implement here, but
as a quick stopgap measure I wanted to try out how much faster we would
go if we didn't need to reestablish a GCloud identity for each file.
CHANGELOG_BEGIN
CHANGELOG_END
run-full-compat: true
Process:
- `git ls-files | xargs sed -i 's/DAML/Daml/g'`
- `git add -p`
To review:
- Check for false positives by carefully reviewing the diff in this PR.
- Check for false negatives with `git grep DAML`.
CHANGELOG_BEGIN
- Removed all traces of DAML. This may affect error messages and debug
outputs.
CHANGELOG_END
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.
I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.
CHANGELOG_BEGIN
CHANGELOG_END
The "tell slack about failures" step is conditional on the branch name
being main; for whatever reason if we don't supply a branch name Azure
sets it to `main`, regardless of the fact that the commit is not in that
branch at all.
CHANGELOG_BEGIN
CHANGELOG_END
It should now be possible for anyone to trigger the daily compat build
by adding a `run-full-compat: true` trailer to their commit message.
CHANGELOG_BEGIN
CHANGELOG_END
run-full-compat: true
* CI: Set `PROJ_DIR` inside the bash lib, not outside.
We can't reference variables set outside.
CHANGELOG_BEGIN
CHANGELOG_END
* CI: Remove an unnecessary `export`.
The existing public key is set to expire in May, so we've changed it.
Note: this _should_ require no other change as the private key is
unchanged (i.e. the new public key can be used to verify old
signatures), but my understanding of GPG is somewhat limited so 🤷.
CHANGELOG_BEGIN
CHANGELOG_END
Three issues here:
1. The release job runs on an Azure-hosted agent, so it doesn't have the
`reset_caches.sh` script (and doesn't need it).
2. The `bash-lib` step should not run if the current job has already
failed.
3. The `skip-github` jobs should also not run if the job has failed.
CHANGELOG_BEGIN
CHANGELOG_END
Two quick improvements I made while waiting on #9039:
- Avoid loading Java. When looking at the logs flow by this seemed to be
taking a huge amount of time.
- Isolate the gcloud config files, which allows for running gcloud
downloads in parallel.
Together these reduce the `check_releases` runtime from about 5 hours to
about 2. There's much more (and smarter) work needed on this, but this
was really easy to do.
CHANGELOG_BEGIN
CHANGELOG_END
My goal here is to investigate the new warning Azure has been showing
for the past few days:
> ##[warning]%25 detected in ##vso command. In March 2021, the agent command parser will be updated to unescape this to %. To opt out of this behavior, set a job level variable DECODE_PERCENTS to false. Setting to true will force this behavior immediately. More information can be found at https://github.com/microsoft/azure-pipelines-agent/blob/master/docs/design/percentEncoding.md
As far as I'm aware we are not deliberately passing in any `%25` in any
of our `vso` commands, so I was a bit surprised by this.
CHANGELOG_BEGIN
CHANGELOG_END
As we strive for more inclusiveness, we are becoming less comfortable
with historically-charged terms being used in our everyday work.
This is targeted for merge on Dec 26, _after_ the necessary
corresponding changes at both the GitHub and Azure Pipelines levels.
CHANGELOG_BEGIN
- DAML Connect development is now conducted from the `main` branch,
rather than the `master` one. If you had any dependency on the
digital-asset/daml repository, you will need to update this parameter.
CHANGELOG_END
The previous escaping syntax doesn’t seem to work (after the first
layer you no longer have a variable) and has broken our build
reporting completely.
changelog_begin
changelog_end
The current implementation of the notify_user job sometimes reports
success while the build has actually failed. Azure does not provide a
way to query the overall state of the current build, so a general
solution to this problem does not seem possible (see #6796 for an
example of a failed attempt). However, all reported cases were
specifically about the `check_changelog_entry` job, which we can easily
query for, so this PR does that.
Note: originally pushed without a changelog entry to test new
notification mechanism.
CHANGELOG_BEGIN
CHANGELOG_END
Currently the return code of the function is the return code of the
`eval "$restore_trap"` line, whereas semantically we want the return
code of the `gsutil` call. This is not an issue in most cases as the
`set -e` should kick in, but if the function appears as the condition in
an `if` statement the `-e` flag is suspended.
The main use-case right now is that the daily license check is _not_
uploading artifacts.
CHANGELOG_BEGIN
CHANGELOG_END
This PR extends the existing `save_gcp_data` function to handle any
`gsutil` command. This is done to support existence checking using
`gsutil ls` for private artifacts in release checking (`ci/cron`).
CHANGELOG_BEGIN
CHANGELOG_END
This is a first, very incomplete step in the spirit of small,
incremental PRs. Known missing features:
- Should check all versions, not just the 30 most recent ones.
- Should also download from GCP backup and compare.
- Should alert on Slack if anything is unexpected.
- Should handle versions prior to us starting to sign (and do what?).
- Should also check artifacts in Artifactory, not just GitHub Releases.
- Optionally should save to GCP if we don't have a backup already.
So at the moment it's just downloading the artifacts for the 30 most
recent releases and printing a message stating whether we have a
signature and whether it's valid.
CHANGELOG_BEGIN
CHANGELOG_END
It's been pointed out to me that some people actually plan their work,
and for them, it would be useful to know at least a day in advance that
they will be doing a release. This PR attempts to accommodate that.
Note: this will run as a new, separate cron. I'll do the Azure setup
after this gets approved.
CHANGELOG_BEGIN
CHANGELOG_END
On the last release, the job succeeded despite no being able to create
the compat PR. This fixes:
- The curl call to actually return non-0 on non-2xx HTTP response.
- The way in which we encode the credentials.
This also attempts to create a Bash library, hopefully this time in a
way that doesn't get destroyed by our release process. IIUC pipeline
instructions (YAML files) are all parsed and read before any execution,
so by embedding the Bash library in a template we should get the correct
version (i.e. the one that is running the pipeline) even when checking
out other commits.
CHANGELOG_BEGIN
CHANGELOG_END
