Commit Graph

774 Commits

Author SHA1 Message Date
mziolekda
576560428f
remove non-repudiation (#18089)
run-all-tests: true
2024-01-10 20:23:04 +00:00
Paul Brauner
bef1938a21
fix calls to open_pr (#18129) 2024-01-10 16:57:05 +01:00
Paul Brauner
489916a1e2
Fix the canton-2.x code drop (#18125)
When deciding whether it's worth creating a PR, diff against the main-2.x branch, not the main one.
2024-01-10 14:19:55 +00:00
Gary Verhaegen
e67f988baf
fix daily snapshot for 2.x (#18123) 2024-01-10 11:39:55 +00:00
Paul Brauner
2a77ae5d0d
Use different branch names for the two code drops (#18120)
The code drop task targeting main-2.x uses the same feature branch name for its PR as the task targeting main. First one wins. This PR fixes that.
2024-01-10 10:30:51 +01:00
Gary Verhaegen
546211105f
only notify canton if upgrade build fails (#18114) 2024-01-10 06:46:38 +01:00
Paul Brauner
54009a41e6
also code drop on main-2.x (#18111) 2024-01-09 18:20:53 +01:00
Gary Verhaegen
bab51e6304
main-2.x: run daily-snapshot job (#18109) 2024-01-09 16:30:31 +01:00
Gary Verhaegen
559a633557
enable prs builds on PRs targeting main-2.x (#18105) 2024-01-09 16:08:17 +01:00
Gary Verhaegen
0b5c4b2a08
remove hourly canton-3x build (#18085)
We want to run this daily instead, which already happens as part of the
daily canton bump PR.
2024-01-04 19:07:41 +01:00
Gary Verhaegen
d48f3130ea
tell team-canton-notifications on canton update failures (#18086) 2024-01-04 19:07:31 +01:00
Remy
ea4085bd28
do not build canton as separate task (#18058) 2023-12-21 09:47:48 +01:00
Paul Brauner
56018b5d6e
do a proper canton 3.x code drop in the canton-3x directory (#17980)
* do a proper canton 3.x code drop in the canton-3x directory

* copy the code from canton3

* address Gary's comments

* fix canton-3x
2023-12-06 10:47:47 +01:00
azure-pipelines[bot]
aa67b0bf30
bump blackduck script to bee7a8c6 (#17984)
CHANGELOG_BEGIN
CHANGELOG_END

Co-authored-by: Azure Pipelines Daml Build <support@digitalasset.com>
2023-12-06 10:47:10 +01:00
dylant-da
40b5eb8476
Fix split release for deps (#17968)
* Resolve release version for sdk build checks

* lint

* lint

run-full-compat: true

* Simple test for using daml-script in release versions

* Fix build issues in tests using pSdkVersion

run-full-compat: true

* Fix build issues with DamlcIntegration

* fix bad sdk version being an invalid version

run-full-compat: true

* Fix the linux "mmap 4096 bytes at (nil): Cannot allocate memory" error

* Fix compat tests on Windows

run-full-compat: true

* test windows os correctly

run-full-compat: true

* temporarily disable canton_3x

run-full-compat: true

---------

Co-authored-by: Paul Brauner <paul.brauner@digitalasset.com>
2023-12-05 23:46:27 +01:00
Paul Brauner
1794212369
build canton 3x in the CI (#17966)
* build canton 3.x in build.yml

* Apply suggestions from code review

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* Apply suggestions from code review

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>

* apply suggestions

---------

Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2023-12-05 15:18:35 +00:00
Paul Brauner
e22f8bebd9
Run costly tests after only after merging (#17956)
* do not run pr-only tests on main, do not run main-only tests on prs

* split data dep tests into main-only and pr-only

* run non-dev conformance tests on main only
2023-12-04 10:52:33 +01:00
tudor-da
e9c4feb562
Make ci/build-canton-3x.sh work on MacOS by managing dir exclusions (#17916)
* Make ci/build-canton-3x.sh work on MacOS by managing dir exclusions

* No need to add the cloned sources to git
2023-11-27 10:08:58 +00:00
Paul Brauner
18e0e9e656
Test that canton 3.x builds with bazel hourly (#17876)
* introduce a canton2 directory

* rename to canton-3x and only keep the bazel rules

* add canton-3x to bazelignore

* fix visibility rules

* add a build-canton-3x script

* add build-canton-3x.sh temporarily to prs.yml to test it out

* try sending a message to stack when failing

* backport 243125adee

* restore prs.yml and add the job to azure-cron.yml

* restore tell-slack-failed condition

* address comments

* display the revision at which canton was cloned, to ease debugging
2023-11-23 08:54:35 +01:00
Paul Brauner
ca31b1beab
restrict code drop and canton jars to 2.x (#17882) 2023-11-22 11:19:22 +01:00
Paul Brauner
f7dd1e6124
Stop downloading the canton opensource jar (#17877)
* stop downloading the canton opensource jar

* udpate obsolete README

* fix one remaining mention of @canton

* remove all mentions of canton_dep.bzlP
2023-11-22 08:36:41 +00:00
Paul Brauner
e4d8609515
try to reset the windows cache (#17825) 2023-11-15 02:05:42 +00:00
Gary Verhaegen
444536ba24
fix daily canton bump (#17781) 2023-11-08 16:30:00 +00:00
Paul Brauner
313b789731
Bump netty and grpc-netty (#17770)
* pin dependencies to json and add missing dep

* fix cyclic dep

* remove unused dep

* add missing dep to //ledger-api/testing-utils:testing-utils

* remove unused dep in //ledger/ledger-api-auth:ledger-api-auth

* remove more unused deps

* more dep fixes

* yet more dep fixing

* more fixing..

* more of the same

* hopefully the last deps to fix

* Bump the version of protobuf and fix everything that depends on it. Took shortcuts that I need to fix in a next commit, but would like to run the CI on this now that it compiles

* don't error out in the grpc-haskell patch

* remove obsolete patch

* patch absl to compile on mingw

* Add a patch to recognize the compiler

* Define _DNS_SD_LIBDISPATCH for macOS gRPC

* bump netty_tcnative_version according to https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty

* pin maven deps

* Fix macos linking errors 'dyld[xxx]: missing symbol called'

* Skip Darwin frameworks in package-app.sh

* pin stackage packages

* pin stackage windows deps

* use the netty version agreed on

* bump the windows global cache to try and debug the upb issue

* restart the CI after timeout

* clean up

* disable failing tests for now

* comment out unused code

* reset the windows machine name to 'default'

---------

Co-authored-by: Moisés Ackerman <6054733+akrmn@users.noreply.github.com>
2023-11-07 19:35:50 +01:00
Gary Verhaegen
442c57b5f4
docker: switch to jammy (#17766) 2023-11-07 12:39:32 +01:00
Remy
44169064ea
fix update_canton_ jobs (#17762) 2023-11-07 11:51:45 +01:00
Gary Verhaegen
00b28969b2
re-start publishing daml-sdk image (#17760)
Turns out some people depend on it. I still think they shouldn't, and we
should work with them to help them move away, but short-term the right
thing to do is to not block their upgrade.
2023-11-06 17:58:23 +01:00
Remy
dc0b2f71a6
mege bump_canton and update_canton_sources jobs (#17757) 2023-11-06 17:37:31 +01:00
Gary Verhaegen
28eadc6e69
fix CODEOWNERS & related files (#17579) 2023-10-16 18:37:14 +02:00
Remy
91b965bd8c
add readme to canton code drop (#17560) 2023-10-16 12:59:20 +02:00
Gary Verhaegen
4a84d2de3c
canton_drop: fix canton credentials (#17551) 2023-10-11 16:13:26 +00:00
Remy
0823af627e
fix name of canton update job (#17544) 2023-10-11 13:12:13 +02:00
Remy
1f31feac2b
Code drop of canton community (#17540)
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2023-10-10 11:29:37 +00:00
Paul Brauner
158cdd90d7
Fix the external name of daml3-script.dar to mention 2.dev (#17512)
* Fix the external name of daml3-script.dar to mention 2.dev

* make the compilation and distribution of daml3-script dars more future proof

* fix path in ci/copy-unix-release-artifacts.sh
2023-10-03 09:16:00 +00:00
Remy
623f305e9c
Move canton directory to let place for code drop (#17506) 2023-09-29 14:37:01 +00:00
Gary Verhaegen
33929d12ae
fix canton bump (#17494)
We've got a few failures recently that report not having delete access
on GCS. This should address those.
2023-09-27 15:22:44 +02:00
Gary Verhaegen
c1280e4439
run blackduck when intended (#17454)
That is, on the daily run and on blackduck-bump PRs.
2023-09-21 22:50:10 -04:00
azure-pipelines[bot]
7d74dc0aa9
bump canton to 2.8.0-snapshot.20230919.11183.0.v52763301 (#17457)
* bump canton to 2.8.0-snapshot.20230919.11183.0.v52763301

CHANGELOG_BEGIN
CHANGELOG_END

* fix canton url

---------

Co-authored-by: Azure Pipelines Daml Build <support@digitalasset.com>
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2023-09-20 18:39:59 +02:00
Gary Verhaegen
3d08291ae6
fix typo in bump-canton job (#17435) 2023-09-19 10:36:14 +00:00
Gary Verhaegen
82655f2f29
keep stable copies of Canton artifacts (#17415) 2023-09-18 18:16:24 +02:00
Gary Verhaegen
8494cff36a
blackduck update: do not fail if branch already exists (#17417) 2023-09-18 16:25:05 +02:00
azure-pipelines[bot]
78487e18a8
fix blackduck update PR run (#17402) 2023-09-15 15:42:44 +02:00
Gary Verhaegen
2dbfb68080
fix BlackDuck scan (#17400)
The latest udpate of the BlackDuck script broke our run. In order to get
a successful scan tomorrow, this PR:

- Pins us back on yesterday's script version, and
- Adds an auto-update mechanism.

This way, we get to stay reasonably up-to-date with automated update
PRs, but we also get to choose when we upgrade.
2023-09-14 16:29:39 -04:00
Gary Verhaegen
4bf7693f42
fix cache (#17377)
Looks like the issue was Bazel 5.2.0.
2023-09-08 10:31:42 +02:00
Gary Verhaegen
675a325cd3
fix Linux cache (macOS still seems broken) (#17369) 2023-09-06 21:21:56 +02:00
Gary Verhaegen
a87818464e
fix bazel cache use on CI (#17363) 2023-09-05 16:51:18 +00:00
Moisés Ackerman
6ae0e0111e
Upgrade to Bazel 5 (again) (#17319)
* Update to rules_haskell v0.16
* Update comments re bazel patches
* clean up bazel overrides
* Upgrade to Bazel 5.2.0
* Remove '--distinct_host_configuration=false'
* Update buildifier to 6.3.2
* Suffix macos and ubuntu caches with yyyymm
* bump windows cache to v14
* [REVERTME] bump linux/macos/darwin timeout to 4h
2023-09-01 15:15:36 +02:00
Samuel Williams
6146149a22
Expose daml3-script to sdk (#17291)
* Expose daml3-script to sdk

* Add warning to daml3-script
2023-08-24 10:42:47 +01:00
Samuel Williams
9179b2e7d0
Jar licensing (#17299)
* Add _distribute.jar target with correct license
Use it everywhere

* Fixes from review

* Change NOTICES to NOTICES.txt for consistency
2023-08-22 15:56:29 +01:00
Paul Brauner
21e8a5408e
Update slack_user_ids (#17251)
Add myself to daml/ci/slack_user_ids
2023-08-09 13:59:06 +02:00
Remy
ae596d7398
Revert "Disable temporary windows build" (#17192)
This reverts commit 908a39e55a.
2023-07-28 09:47:00 +00:00
Remy
9f9e25168d
Disable temporary windows build (#17186)
until we have windows back in the pipeline
2023-07-27 09:05:15 +00:00
Raphael Speyer
47d21fe717
JSON API: add PruningTest and make it pass (#17167)
If the ledger has been pruned more recently that the last cached copy, then attempting to fetch the changes since that last offset will fail, rending the relevant template(s) unqueryable. This PR detects that condition, clears the cache for the relevant template and queries again, which will refresh the cache with a fresh copy of the ACS for that template, and serve the query from that.

I also made some usability tweaks around running canton-ee tests, to help improve the dev experience for failures I came across while trying to run them. Specifically
* Use `--config=canton-ee` to opt into the tests which require canton-ee
* When downloading that EE from artifactory, provide better diagnostics if the required auth isn't set up.
2023-07-27 09:12:22 +10:00
Gary Verhaegen
d03ffb0329
fix bump_canton (#17166) 2023-07-24 13:19:05 +02:00
Gary Verhaegen
eda4698ced
enable manual cnaton bumps (#17155) 2023-07-20 14:59:26 +02:00
Gary Verhaegen
03a2d336d2
fix local canton for compat (#17146) 2023-07-19 12:18:58 +00:00
Gary Verhaegen
a6cb39d545
release: notify canton channel (#17151) 2023-07-19 14:04:32 +02:00
Gary Verhaegen
c71b781cc6
stop publishing daml-sdk image (#17132) 2023-07-17 17:21:12 +02:00
Samuel Williams
f82fb77102
Update tests to match canton changes (#17093) 2023-07-12 16:53:32 +01:00
Gary Verhaegen
af9ce9fc8e
delete old logs before starting (#17117) 2023-07-12 10:08:20 +00:00
Gary Verhaegen
3b9951bc7f
run m1 sometimes (#17099) 2023-07-12 11:31:38 +02:00
Gary Verhaegen
0374acb775
ci: compress results on failure (#17108)
Right now logs only get compressed on successful runs, which is not
necessarily when they are smallest.
2023-07-11 17:57:55 +02:00
Gary Verhaegen
242fe0f447
add job attempt to logs (#17091)
For the used-to-be-rare-but-not-so-much-anymore case where the job fails
after having pushed its logs (without this the push fails as we can't
overwrite artifacts).
2023-07-06 15:35:49 +00:00
Gary Verhaegen
53556dad06
ci: run fewer things on m1 (#17092)
This was spurred by the fact that the "report_end" task sometimes fails
on m1 with the "install Bash lib" step just never finishing (and the
whole job then times out after 6h).

Hopefully by running fewer things we get fewer chances of these kinds of
weird issues.

Note that it's unclear if anything actually crashes on the m1 machines
or if this is a loss of connection between Azure Pipelines and the
machine. From what I've seen as soon as that job times out the machine
is able to successfully pick up other jobs. Speaking of, I've also
reduced the 6h timeouts to a more reasonable 3h.
2023-07-06 16:55:39 +02:00
Gary Verhaegen
fbe4d0de1f
ci: rerun m1 three times (#17089)
The m1 build is extra flaky, maybe running it multiple times will help.
2023-07-06 16:42:20 +02:00
Gary Verhaegen
f6a78df1c8
ci: compress Bazel logs (#17090)
We routinely have upwards of 3GB of logs. They are very rarely
downloaded, most people don't even know they're there. Uploading 3GB
takes time. This should make it faster, hopefully.
2023-07-06 16:36:00 +02:00
Gary Verhaegen
cace97977d
fix bump_canton (#17062)
run-full-compat: true
2023-06-29 12:20:38 +00:00
Remy
9e0493cc1a
add support for Canton EE available to JSON API (#17034)
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
2023-06-29 09:03:32 +00:00
Gary Verhaegen
0e9cb10f7d
optionally get canton EE (#17039) 2023-06-27 11:00:24 +02:00
Gary Verhaegen
f6a51dd083
clean up refresh-get-daml (#16994) 2023-06-15 09:49:07 +01:00
Gary Verhaegen
ed202d1861
refresh-get-daml-com: add invalidation & format (#16993)
* refresh-get-daml-com: add invalidation & format

* test update
2023-06-14 18:54:52 +00:00
Gary Verhaegen
9fadea3f4c
fix get-daml.sh update script (#16992) 2023-06-14 16:58:45 +02:00
Gary Verhaegen
3cadcc0679
fix YAML (#16991) 2023-06-14 12:09:58 +00:00
Gary Verhaegen
2c53e5a648
fix main ci (#16986) 2023-06-13 16:38:36 +00:00
Gary Verhaegen
d75ca23db5
update get-daml-com (#16985)
So far updates to the result of `curl https://get.daml.com` have been
made manually. This automates the process.

Fixes #3915.
2023-06-13 15:36:32 +00:00
Gary Verhaegen
77294d35ed
fix bump-canton job (#16959)
run-full-compat: true
2023-06-07 13:41:48 +02:00
Gary Verhaegen
a791df7c1c
fix canton build (#16954)
It's not all that useful if we don't actually build the commit we're
trying to build.
2023-06-05 10:58:11 +00:00
Gary Verhaegen
de6f31c4d8
canton: allow depending on arbitrary sha (#16926) 2023-06-01 15:06:35 +00:00
Remy
4845ddf48d
Skip 1.dev integration tests on Windows (#16937) 2023-05-30 14:42:32 +00:00
Gary Verhaegen
8d0357ce02
update bump_canton for new snapshot format (#16919) 2023-05-25 17:05:00 +02:00
mziolekda
06abd5c649
Clean up ledger dependencies (#16894)
* remove sandbox-on-x project

update bazel readme

update release artifacts

comment out last remaining SoX test

remove ledger-runner-common

remove participant-state-kv-errors

remove recovering-indexer-integration-tests

remove participant-integration-api

update doc pages

cleanup ledger-api-auth usage

remove participant-state

fix build

fix build

clean up ledger-api-common part I

clean up ledger-api-comon part II

clean up ledger-api-common part III

remove ledger/metrics

clean up ledger-api-health and ledger-api-domain

* remove ledger-configuration ad ledger-offset

* remove ledger-grpc and clean up participant-local-store

* reshuffle few more classes

* format
2023-05-23 13:44:17 +00:00
mziolekda
95cc249ddd
Remove sandbox on x (#16890)
* remove sandbox-on-x project

* update bazel readme

* update release artifacts

* comment out last remaining SoX test

* remove ledger-runner-common

* remove participant-state-kv-errors

* remove recovering-indexer-integration-tests

* remove participant-integration-api

* update doc pages

* cleanup ledger-api-auth usage

* remove participant-state

* fix build

* fix build
2023-05-23 09:25:54 +02:00
Remy
b2eb3224e7
fix canton update hook (#16851) 2023-05-12 09:13:07 +00:00
Remy
a863d69d34
Move CatonFixture in its own bazel module (#16824) 2023-05-10 23:15:26 +00:00
Gary Verhaegen
d37fe6f7e8
fix canton switch (#16785) 2023-05-02 14:21:19 +00:00
Gary Verhaegen
1f3c22cf10
reenable canton tests every day (#16765)
This is obviously not going to do anything until some tests have the
tag.
2023-04-27 21:42:16 +00:00
Gary Verhaegen
3d9a8b3084
revert oracle image bump (#16682)
Reverts "update oracle test image (#16650)".
2023-04-13 13:54:09 +00:00
mziolekda
b446fba6d8
update oracle test image to digitalasset/oracle:enterprise-19.18.0-preloaded-20230403-35-6f80341 (#16650) 2023-04-10 13:40:47 +02:00
Gary Verhaegen
f3943c4574
fix canton update (#16622)
Broken by #16595.
2023-03-29 17:03:16 +02:00
Gary Verhaegen
7f6b1fe3e8
Fix blackduck (#16598)
* fix blackduck

BlackDuck is currently failing to run because it creates multiple report
files and then expects a single one. This is due to #16552 changing only
one of the output file names.

This PR resolves this by:

1. Changing the name everywhere, so all runs aggregate their results on
   the same file.
2. Changing the PR step code to expect the given file name, rather than
   try and glob it.

My understanding of the BlackDuck setup is very fuzzy so please
double-check carefully.

See #16600 for an example of what the update would be from this PR.

* Update daily-compat.yml

main instead of local-main-maven

---------

Co-authored-by: Brian Healey <brian.healey@digitalasset.com>
2023-03-27 00:31:25 +00:00
Gary Verhaegen
d2664edbf1
fix bump_canton (#16558)
Broken by #16538 yesterday.

run-full-compat: true
2023-03-16 11:59:52 -04:00
Brian Healey
81107ff834
correct to --detect.bazel.workspace.rules to fix bazel JVM and Haskell scans (#16552) 2023-03-15 14:26:36 -04:00
Gary Verhaegen
4fa880a5fc
don't run fix-bazel-cache on PRs (#16377)
I'm annoyed at how slow it runs. The hourly cron should be good enough.
2023-02-23 15:59:57 +01:00
Gary Verhaegen
7f36d5e4b1
[docs] remove manually-edited rst files (#16376)
They are moving to the docs repo. I'm removing them here first and will
add to the docs repo afterwards, to avoid losing edits in the meantime.
2023-02-23 15:55:04 +01:00
Gary Verhaegen
8435730ac3
rewrite check-releases in Bash (#16260) 2023-02-15 11:11:41 +01:00
Remy
61eae5c3d3
[compatibility test] update to LF 1.15 (#16259) 2023-02-10 09:10:16 +01:00
Gary Verhaegen
fc8eadf164
stop managing docs (#16226)
The docs S3 bucket will now be entirely managed by the docs repo CI.
2023-02-06 13:29:03 +01:00
Gary Verhaegen
89f0dad016
cron: fix sgnature key (#16115)
I _believe_ this is what has been tripping up the check-releses job for
the past few days: while updating the signing key, I did not update the
key fingerprint to match, and therefore trying to check any signature
fails.

The failure mode of the existing script is still pretty bad, regadless:
looping on the same release until the hard drive is full is definitely
not expected. Our retry policies are supposed to have time bounds, and
failure should result in cleaning up the workdir.

run-full-compat: true
2023-01-23 10:48:04 +00:00
Gary Verhaegen
1fd0c5bf2d
update docs to mention the new key (#16042)
Note: this is only going to affect 2.6.0 docs and later; the existing,
published documentation will be updated manually to match.
2023-01-11 21:13:16 +00:00
Gary Verhaegen
79058cc4fa
comment COPY in canton_dep (#16007) 2023-01-05 16:47:16 +01:00