* Resolve release version for sdk build checks
* lint
* lint
run-full-compat: true
* Simple test for using daml-script in release versions
* Fix build issues in tests using pSdkVersion
run-full-compat: true
* Fix build issues with DamlcIntegration
* fix bad sdk version being an invalid version
run-full-compat: true
* Fix the linux "mmap 4096 bytes at (nil): Cannot allocate memory" error
* Fix compat tests on Windows
run-full-compat: true
* test windows os correctly
run-full-compat: true
* temporarily disable canton_3x
run-full-compat: true
---------
Co-authored-by: Paul Brauner <paul.brauner@digitalasset.com>
* build canton 3.x in build.yml
* Apply suggestions from code review
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* Apply suggestions from code review
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* apply suggestions
---------
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* do not run pr-only tests on main, do not run main-only tests on prs
* split data dep tests into main-only and pr-only
* run non-dev conformance tests on main only
The latest udpate of the BlackDuck script broke our run. In order to get
a successful scan tomorrow, this PR:
- Pins us back on yesterday's script version, and
- Adds an auto-update mechanism.
This way, we get to stay reasonably up-to-date with automated update
PRs, but we also get to choose when we upgrade.
* Update to rules_haskell v0.16
* Update comments re bazel patches
* clean up bazel overrides
* Upgrade to Bazel 5.2.0
* Remove '--distinct_host_configuration=false'
* Update buildifier to 6.3.2
* Suffix macos and ubuntu caches with yyyymm
* bump windows cache to v14
* [REVERTME] bump linux/macos/darwin timeout to 4h
If the ledger has been pruned more recently that the last cached copy, then attempting to fetch the changes since that last offset will fail, rending the relevant template(s) unqueryable. This PR detects that condition, clears the cache for the relevant template and queries again, which will refresh the cache with a fresh copy of the ACS for that template, and serve the query from that.
I also made some usability tweaks around running canton-ee tests, to help improve the dev experience for failures I came across while trying to run them. Specifically
* Use `--config=canton-ee` to opt into the tests which require canton-ee
* When downloading that EE from artifactory, provide better diagnostics if the required auth isn't set up.
This was spurred by the fact that the "report_end" task sometimes fails
on m1 with the "install Bash lib" step just never finishing (and the
whole job then times out after 6h).
Hopefully by running fewer things we get fewer chances of these kinds of
weird issues.
Note that it's unclear if anything actually crashes on the m1 machines
or if this is a loss of connection between Azure Pipelines and the
machine. From what I've seen as soon as that job times out the machine
is able to successfully pick up other jobs. Speaking of, I've also
reduced the 6h timeouts to a more reasonable 3h.
We routinely have upwards of 3GB of logs. They are very rarely
downloaded, most people don't even know they're there. Uploading 3GB
takes time. This should make it faster, hopefully.
In #14622, I inadvertently changed the artifact names from using `macos`
to using `macOS`, which may break any number of external references,
from Canton's dependency (which is how I found out) to the assembly
script and the `daml install` command. This reverts that.
CHANGELOG_BEGIN
CHANGELOG_END
This adds parameters to the "split release" job. At the moment, when
run, the job just builds the current commit with a snapshot version.
With this change, we'll be able to build a split release for any commit
with any version tag.
The reason for this change is we want to trigger this job from the
assembly repo, to reduce the number of manual interventions needed in
the release process.
CHANGELOG_BEGIN
CHANGELOG_END
Azure is telling us to stop using 2019, so I've updated the job to be
compatible with 2022. (And future Windows images, too.)
CHANGELOG_BEGIN
CHANGELOG_END
Since #12645, we added a new pipeline, so we need to add a corresponding
entry.
As for #12645, the content of the files and the directory structure is
taken directly from a live CI node, as printed by the (now-named)
`workdirs` step.
CHANGELOG_BEGIN
CHANGELOG_END
This reverts a small part of 9e1e42d27c
(#12901).
Microsoft seems to have given us our free parallel jobs back, so we can
run the patch_bazel job again.
Fixes#12900.
CHANGELOG_BEGIN
CHANGELOG_END
This should hopefully get CI working again.
There are two changes in here:
1. We can no longer change our patched Bazel. I didn’t switch away
from the current patched version for now (we upload it to gcp bucket
so it still works fine even if we cannot build it) but if we upgrade,
we need to go to an unpatched version for now.
2. We need to get `az` from dev-env. I tested the self service compat
job stuff and it works fine with this but there is a chance other
parts don’t.
changelog_begin
changelog_end
Because there's no reason not to. The only obstacle to automating the
normal release process is that we need an explicit manual validation
step for our audit log when creating a release, but split-releases are
created in the Assemblty repo, so we can have the audit log over there.
The diff is going to be very messy because there's a lot of stuff moving
around. The only thing that is not just moving a job to a separate file
is the `ci/daily-snapshot.yml` file, which is 100% new and is meant as
a new Azure Pipelines entrypoint (which I will create after this gets
approved). I have made a reasonable effort to create individual commits
that simplify reviewing, but I expect it's still going to be kind of a
mess. I'm open to opening separate PRs to ~bump my stats~ move one job
at a time if that makes reviewing (and testing) easier.
CHANGELOG_BEGIN
CHANGELOG_END
The mapping from Azure job to local folder is managed by a handful of
JSON files under `$WORKDIR/SourceRootMapping`. Based on the sketchy
[documentation] that screams "don't mess with these definitely internal,
subject-to-change files", it seeems to me that our best bet for pinning
Windows work folders is to mess with these files. Specifically, take a
"version" of these files we like from a live CI machine and add those
specific files to the init script of our Windows nodes.
[documentation]: https://github.com/microsoft/azure-pipelines-agent/blob/master/docs/jobdirectories.md
So the first step is to collect these files, which is what this PR aims
to accomplish.
CHANGELOG_BEGIN
CHANGELOG_END
I'm not quite sure what's going wrong here, but this PR:
- Fixes a small bug in the process for determininng that we're _not_ in
a split release, which worked because the boolean expression crashing
is the same as the boolean expression returning false. But I'd still
prefer if it returned false without crashing.
- Changes the condition for skipping the Windows installer so that in
case of ternary+ logic we default to the harmless behaviour (including
it).
- Makes a change that might have some impact maybe on the way Azure
expands templates which may or may not result in the installer being
excluded when it should be excluded. But this time if it doesn't work
worst case we get an extra unused binary that uses some disk space,
instead of crashing the release.
CHANGELOG_BEGIN
CHANGELOG_END
In the split release process, the executable installer has to be created
later in the pipeline, once we have the Canton parts available.
CHANGELOG_BEGIN
CHANGELOG_END
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.
I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.
CHANGELOG_BEGIN
CHANGELOG_END
* Split release process (PoC)
This PR adds the necessary infrastructure for the new split release
process. Releases are still triggered via the LATEST file but you can
choose between the old and the new release process by adding
SPLIT-RELEASE at the end of the line.
As a first step this publishes all artifacts we currently publish to
Github releases to our GCP bucket.
changelog_begin
changelog_end
* drop report-start
changelog_begin
changelog_end
* fix gcp bucket
changelog_begin
changelog_end
* review feedback
changelog_begin
changelog_end
* Update azure-pipelines.yml
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* SPLIT-RELEASE ->SPLIT_RELEASE
changelog_begin
changelog_end
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
