* webide: build webide image when sdk releases
* add scripts which check the latest version of sdk. If webide docker
image version does not exist or is older than the sdk version, it will
kick off a build of the webide docker image
* add job to azure cron
* webide: minor response to review
* windows: fixed daml-lf tests for Windows by using Bazel's rlocation
* more consistent logging on CI; publishing Windows test logs on failure
* windows: fix daml-lf engine tests
* windows: add diff tool to msys
This is a first step towards improving our docs release process. The
goal here is to get rid of the manual "publish docs" step. This is done
as a periodic check because we only want to run this for "published"
releases, i.e. the ones that are not marked as prerelease. Because the
act of publishing a release is a manual step that Azure cannot trigger
on, we instead opt for a periodic check.
Not included in this piece of work:
- Any change to the docs themselves; the goal here is to automate the
current process as a first step. Future plans for the docs themselves
include adding links to older versions of the docs.
- A better way to detect docs are already up-to-date, and abort if so.
- Including older versions of the docs.
- Switching the DNS record from the current AWS S3 bucket to this new
GCS bucket. That will be a manual step once we're happy with how the
new bucket works.
This reverts commit 3d8acde916.
For some reason that commit seems to have resulted in a lot of
"unexpected end of file" errors during cache downloads. I do not know
what is going on here or how to fix it so let’s revert it for now.
* release: make 'ci/release.sh' runnable for dry runs.
release-dry-run.sh is outdated and duplicates logic from ci/release.sh, so it
got deleted.
* ledger-api-test-tool: release the tool together with the SDK components.
* ledger-api-test-tool: update docs to reflect distribution mechanism.
* ledger-api-test-tool: further docs refinements.
* Add Ledger API Test Tool mention into release notes.
The ci/release.sh fails if the BUILD_SOURCEBRANCHNAME environment
variable is not set. Although this variable is normally set by the
CI system, it is sometimes useful to run the script manually and
simply adding an 'invalid' default to the check of the env variable
means that the script still works if the variable is unbound.
The newer version seems to segfault on MacOS quite often so let’s
downgrade for now. We should also try to see if we can find a
reasonable way of reproducing this and report it upstream.
As multiple platforms will create different annotated tags (because an
annotated tag includes a tag time), they will conflict on trying to
push. Therefore, we go for a lightweight tag for now, as those are
simple pointers to a commit and git will recognize that they are the
same and there is no conflict.
This rewrites the release script to be a lot simpler and significantly
faster:
- The artifacts are now declared in a separate yaml file which should
make it easier for people to modify and doesn’t clutter the actual
code.
- There is only a constant number of calls to Bazel which speeds up
the script quite a bit.
I verified that the release artifacts are the same that we got
before and I traced the calls to the jfrog binary in a fake release
and ignoring order they are identical.
This adds `ci/azure-cleanup`, containing a script that talks to azure pipelines, removing agents older than 25 hours in a specific pool.
Machines are meant to be killed after 24 hours anyway, make sure they're properly unregistered from Azure Pipelines, too.
By doing this, we don't need to unregister nodes manually on shutdown.
Idea is to execute this every time a new agent is provisioned, it has cloned the repo. We intend to clone the repo and pre-warm the caches there anyhow.
WIP until the repo fetching and cache pre-warming is present, too.
cc @zimbatm
### Pull Request Checklist
- [x] Read and understand the [contribution guidelines](https://github.com/digital-asset/daml/blob/master/CONTRIBUTING.md)
- [x] Include appropriate tests
- [x] Set a descriptive title and thorough description
- [x] Add a reference to the [issue this PR will solve](https://github.com/digital-asset/daml/issues), if appropriate
- [x] Add a line to the [release notes](https://github.com/digital-asset/daml/blob/master/docs/source/support/release-notes.rst), if appropriate
NOTE: CI is not automatically run on non-members pull-requests for security
reasons. The reviewer will have to comment with `/AzurePipelines run` to
trigger the build.
Azure Pipelines has direct integration with GitHub, so we're just using
that. Releases on GitHub have to target a tag, so we also need to push
the tag as an intermediate step; we also need to include the platform
name in the artifact to avoid overwriting from different builds.
The two "GitHub release" steps depend on two Azure variables that are
not defined in the pipeline script. This may look like it should not
work, but in fact it does, because these variables are set by the
release script.
In Azure Pipelines, any build step can set variables for the next build
steps by outputting specially-formatted text to stdout. This text will
not appear in the build output displayed by Azure Pipelines, e.g.:
```
echo '##vso[task.setvariable variable=sauce]tomatoes'
```
would define the Azure variable `sauce` to have the string `tomatoes` as
its value for the next build steps.
See [0] for details.
[0]: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#set-in-script
* nix: add the more providers to terraform
* docs: make tarballs more reproducible
* ci: use the linux-pool pool
* ci: tweak the nix installation
handle the case where the user is root and on ubuntu
* infra: terraform fmt
* infra: add Azure Pipeline agents
* ci: only enable linux-pool for internal PRs
Without this PR, this variable ends up being set to the string
"variables['System.PullRequest.IsFork']" which meant that we never
uploaded to the Bazel cache.
