Commit Graph

211 Commits

Author SHA1 Message Date
Moisés Ackerman
6072629c7a
Request review for release rotation PR (#14287)
* Merge ci/cron/wednesday.yml open_pr function into ci/bash-lib.yml

* remove unused function reset from ci/cron/wednesday.yml

* Request review from current release person for rotation PR

changelog_begin
changelog_end
2022-06-28 19:45:37 +02:00
Gary Verhaegen
d6dd03751f
cron: faster check_releases (hopefully) (#14209)
The check_releases job has been a major player in the flakiness of the
daily test lately, simply by _timing out_ despite its 6h limit.

There are smarter, more "permanent" fixes we could implement here, but
as a quick stopgap measure I wanted to try out how much faster we would
go if we didn't need to reestablish a GCloud identity for each file.

CHANGELOG_BEGIN
CHANGELOG_END

run-full-compat: true
2022-06-17 17:56:51 +02:00
Gary Verhaegen
e78303332a
docs: fix top-level update function (#14204)
After yet another report of a ghost file, I stared at the code even
harder than before and finally figured out what was going wrong (or at
least one way it was going wrong): `listDirectory` is not recursive, so
the current code would only delete top-level files.

Hopefully this fixes that. I'll try to do some manual cleanup too.

CHANGELOG_BEGIN
CHANGELOG_END
2022-06-16 18:42:39 +00:00
Gary Verhaegen
9a98fa04f5
fix compat bump (#13702)
CHANGELOG_BEGIN
CHANGELOG_END
2022-04-26 15:46:46 +00:00
Gary Verhaegen
474bbe1327
restore compat update (#13663)
With the new 2.0.0 release process, we're not releasing from the main CI
build anymore, so the compat update PRs are not getting created.

This restores that by making those PR creations part of the daily CI
build.

CHANGELOG_BEGIN
CHANGELOG_END
2022-04-22 17:28:16 +02:00
Gary Verhaegen
8a4b2c2844
fix bump-canton (#13626)
- Add notifications so I don't miss te updates.
- Add explicit `fetch` so we can detect if the same branch already
  exists.
- Run from `main` so we geet a fresh view in case of reruns.

CHANGELOG_BEGIN
CHANGELOG_END
2022-04-19 12:27:17 +02:00
Gary Verhaegen
957db9b091
auto-bump canton (#13596)
Or, maybe, break CI trying to.

CHANGELOG_BEGIN
CHANGELOG_END
2022-04-14 17:02:37 +00:00
Gary Verhaegen
90e03a3044
docs cron: do not build post-2.0 (#13572)
After 2.0, the docs need to be built alongside Canton files, and that is
taken care of by the [docs.daml.com] repo.

[docs.daml.com]: https://github.com/digital-asset/docs.daml.com

CHANGELOG_BEGIN
CHANGELOG_END
2022-04-12 13:13:50 +02:00
Gary Verhaegen
19714e27c6
ci/cron/docs: fix file verification (#13475)
There is no `versions.json` in daml-on-sql docs.

CHANGELOG_BEGIN
CHANGELOG_END
2022-03-31 14:25:40 +02:00
Gary Verhaegen
961ca80b77
hunt down DAML (#13191)
Process:

- `git ls-files | xargs sed -i 's/DAML/Daml/g'`
- `git add -p`

To review:

- Check for false positives by carefully reviewing the diff in this PR.
- Check for false negatives with `git grep DAML`.

CHANGELOG_BEGIN
- Removed all traces of DAML. This may affect error messages and debug
  outputs.
CHANGELOG_END
2022-03-08 14:25:25 +01:00
Gary Verhaegen
54dc19deb8
add release instructions to release notification (#13160)
Because we can't expect people to magically know where to find them.

CHANGELOG_BEGIN
CHANGELOG_END
2022-03-04 11:33:15 +00:00
Victor Peter Rouven Müller
0dc167fa48
[JSON-API] Remove dependency on user provided tokens for the perf runner & refactor main (#13114)
* Don't error when user tokens are provided for the perf runner

changelog_begin
changelog_end

* Better token parsing error handling & extract user id to allocate the user if so

* make the code prettier

* Fix & simplify token parsing

* fix formatting of bazel file

* Update ledger-service/http-json-perf/src/main/scala/com/daml/http/perf/Main.scala

Co-authored-by: Raymond Roestenburg <98821776+ray-roestenburg-da@users.noreply.github.com>

* correctly handle the response of createUser & don't throw if no LedgerId was found in the token

* Wrap exceptions in Futures

* Refactor Main.scala of the perf runner completely & remove dependency on user provided JWT's

* Minimize diff

* simplify code further

* Update ledger-service/http-json-perf/src/main/scala/com/daml/http/perf/Main.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Update ledger-service/http-json-perf/src/main/scala/com/daml/http/perf/Main.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Update ledger-service/http-json-perf/src/main/scala/com/daml/http/perf/Main.scala

Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>

* Fix build

* Don't pass a jwt to the perf runner anymore everywhere it was used & fix ledger id to be right

* Minimize diff

Co-authored-by: Raymond Roestenburg <98821776+ray-roestenburg-da@users.noreply.github.com>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
2022-03-03 16:12:40 +00:00
Moritz Kiefer
9e1e42d27c
Drop dependency on hosted nodes (#12901)
This should hopefully get CI working again.

There are two changes in here:

1. We can no longer change our patched Bazel. I didn’t switch away
   from the current patched version for now (we upload it to gcp bucket
   so it still works fine even if we cannot build it) but if we upgrade,
   we need to go to an unpatched version for now.
2. We need to get `az` from dev-env. I tested the self service compat
   job stuff and it works fine with this but there is a chance other
   parts don’t.

changelog_begin
changelog_end
2022-02-14 02:49:38 +01:00
Moritz Kiefer
6a91fdfe22
Fix version check in docs cron (#12882)
2 is not a valid version

changelog_begin
changelog_end
2022-02-10 20:25:04 +00:00
mziolekda
42ff83d456
Remove daml on sql artifacts and docs (#12870)
* stop publishing daml on sql aka Daml Driver for PotgreSQL

CHANGELOG_BEGIN
Stop publishing Daml Driver for PostgreSQL
CHANGELOG_END
2022-02-10 18:01:18 +00:00
Gary Verhaegen
3f2499dd41
stop creating snapshot PRs (#12819)
In the new process we'll rely on nightly split builds instead.

CHANGELOG_BEGIN
CHANGELOG_END
2022-02-08 18:57:23 +01:00
Moisés Ackerman
e4764cc426
Upgrade to GHC 9.0.2 (#12300)
changelog_begin
changelog_end

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2022-02-01 11:27:11 +01:00
Brian Healey
ef18bf4219
DEV all caps needed to exclude dev dependencies (#12679) 2022-01-31 16:39:51 +00:00
Brian Healey
215a63d55c
correct deprecated options with newest synopsys-detect version (#12586)
CHANGELOG_BEGIN
CHANGELOG_END
2022-01-26 08:41:41 +01:00
Moritz Kiefer
d0813e61f1
Factor out version check in docs cron (#12459)
follow up to #12456

changelog_begin
changelog_end
2022-01-18 19:38:47 +00:00
Moritz Kiefer
01b0b76807
Support split releases in the docs cronjob (#12456)
Most of this is just documentation. The actual idea here is fairly
simple:

- The assembly repo pushes to S3 on each release (not yet implemented).
- The cronjob skips releases that are already there.
- Updating the top-level just piggybacks on what we already have.

changelog_begin
changelog_end
2022-01-18 16:05:31 +01:00
Moritz Kiefer
27bfd40365
Split Daml cron executable into separate modules (#12449)
This follows the approach we used for the BazelCache stuff and splits
the 3 different operations in 3 separate modules + one Github utility
module to make it a bit easier to follow.

This is pure reshuffling, no functional change.

changelog_begin
changelog_end
2022-01-18 09:46:29 +00:00
Gary Verhaegen
d2e2c21684
update copyright headers (#12240)
New year, new copyright, new expected unknown issues with various files
that won't be covered by the script and/or will be but shouldn't change.

I'll do the details on Jan 1, but would appreciate this being
preapproved so I can actually get it merged by then.

CHANGELOG_BEGIN
CHANGELOG_END
2022-01-03 16:36:51 +00:00
Brian Healey
dc6fdaaca9
revert back to running latest blackduck detect 6.9.0 (#12177)
* revert back to running latest blackduck detect 6.9.0

CHANGELOG_BEGIN
CHANGELOG_END

run-full-compat: true

* revert back to running latest blackduck detect 6.9.0

CHANGELOG_BEGIN
CHANGELOG_END

run-full-compat: true

* try increasing heap

10gb heap for blackduck scan

run-full-compat: true

* exclude bazel-out

run-full-compat: true

* separate blackduck scan stages
no signature or binary scan

run-full-compat: true

* exclude result- directories

run-full-compat: true

* do not follow symbolic links

run-full-compat: true

* exclude node rules

run-full-compat: true

* exclude node rules

    run-full-compat: true

* do not follow symbolic links for detector runs

run-full-compat: true

* yarn install language-support

run-full-compat: true

* exclude ts dirs

run-full-compat: true

* exclusions

run-full-compat: true

* all dirs

run-full-compat: true

* add go executable

run-full-compat: true

* no need for sig scan properties in go detector scan

run-full-compat: true

* Fixes # 6419 remove not-main for testing

* Update ci/cron/daily-compat.yml

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

* Update ci/cron/daily-compat.yml

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>

Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
2021-12-21 18:08:42 +00:00
Moritz Kiefer
f847767e36
Avoid nix result-* symlinks on CI (#12220)
We really don’t want those anywhere and they currently trip up
blackduck which starts scanning our nix store.

changelog_begin
changelog_end
2021-12-21 07:59:02 -05:00
Brian Healey
bb89f369c0
revert to use synopsys detect 7.1.0 (#12155)
* revert to use synopsys detect 7.1.0 run-full-compat: true

CHANGELOG_BEGIN
CHANGELOG_END

* run on branch run full compat

run-full-compat: true

* run on branch run full compat

run-full-compat: true

* revert changes to run branch
2021-12-15 14:39:28 -05:00
Moritz Kiefer
a3556a57ae
Retry release signature checks (#11771)
changelog_begin
changelog_end
2021-11-18 14:09:42 +00:00
Gary Verhaegen
fcd4549569
test release rotation: factor out selection (#11161)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-07 12:46:55 +00:00
Stefano Baghino
265e36255c
Add a notice to the release testing rotation file (#11142)
Reminds people adding new users to the rotation that they should also
be added to DACH-NY/daml-language-ad-hoc to have a box in the cloud
for Windows testing.

Adds the possibility of adding comments at the beginning of the rotation file.

changelog_begin
changelog_end
2021-10-06 21:08:11 +00:00
Remy
37d94aaef6
Add unit for daily performance tests (#11107)
CHANGELOG_BEGIN
CHANGELOG_END
2021-10-04 13:00:38 +02:00
Gary Verhaegen
6ae3afa8fb
fix perf reporting (#11073)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-29 17:39:46 +00:00
Gary Verhaegen
12e0c72d5c
fix blackduck logic (#11049)
We believe the Blakduck logic is currently faulty. We have had a
violation on an NPM dependency, and Blackduck keeps reporting it despite
our having removed the dependency.

We believe that what is happening is that, in the first step of
checking, we udpate the Haskell dependencies, _and then check the
validity of the whole project_, which includes the NPM deps. Because
that fails, we never get to the step where we actually update the NPM
deps, and Blackduck is stuck forever.

The solution is to not fail on violations for the Haskell update steps.
Haskell deps are still checked in the second step, because, again, it is
checking the whole project.

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-28 14:28:19 +00:00
akshayshirahatti-da
721575ea73
[JSON-API] Postgres perf job (#10986)
* Addition of a postgres perf job

CHANGELOG_BEGIN
CHANGELOG_END

* use a single job with strategy matrix

* cleanup test job from main pipeline
2021-09-23 09:16:58 +01:00
akshayshirahatti-da
fb5ab5dca2
setvar doesn't like new lines in assignment, refactor (#10842)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-10 14:04:24 +00:00
akshayshirahatti-da
7824201b8d
Get rid of extra new line and prune precision for all gatling values (#10820)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-09 08:37:50 +00:00
akshayshirahatti-da
35a853fd30
Async logging for http-json-perf tests and publish logs as a pipeline artifact (#10803)
CHANGELOG_BEGIN
CHANGELOG_END
2021-09-07 23:42:30 +01:00
Gary Verhaegen
a349217f78
get oracle perf test output (#10800)
We can only reference variables set in a named step.

CHANGELOG_BEGIN
CHANGELOG_END
2021-09-07 23:14:44 +02:00
akshayshirahatti-da
f68a12930b
Report some oracle_json_perf numbers on slack (#10754)
* Add avg, stddev, p90, p99, requests_per_second numbers to be reported on slack similar to speedy_perf

changelog_begin
changelog_end

* changes based on code review

* fix failing job due to breaking function export
2021-09-02 17:44:01 +01:00
akshayshirahatti-da
4525b8c265
[JSON-API] vanilla oracle_perf ci job (#10688)
* vanilla job test on main pipeline

changelog_begin
changelog_end

* move job to daily compat tests

* add timeout to dev-env and changes based on code review
2021-09-02 09:31:56 +01:00
Moritz Kiefer
5e7f8e6a49
Handle not quite empty directory digests in cache cleanup (#10536)
Dropping the strict source name patch has resulted in empty
directories which Bazel is happy about but then fail at
runtime. They’re not quite empty, the digest points to 0x0a 0x00
so we match on that.

changelog_begin
changelog_end
2021-08-10 15:06:10 +00:00
Gary Verhaegen
da9f8e3007
clean-up perf tests (#10355)
We've removed some features, this can be simplified a bit now.

CHANGELOG_BEGIN
CHANGELOG_END
2021-07-22 12:07:19 +00:00
Moritz Kiefer
5bf7d8f570
Fix speedy perf reporting (#10100)
jq does not like dashes see https://github.com/stedolan/jq/issues/38

changelog_begin
changelog_end
2021-06-23 20:16:47 +02:00
Moritz Kiefer
c44f33736e
Simplify speedy benchmark (#10079)
This PR drops two things:

1. The check that the benchmark hasn’t been modified. This hasn’t ever
been useful and it keeps being annoying.
2. It stops the comparison against the old version and instead just
benchmarks the current version. We really only care about the day to
day changes. Comparing against an arbitrary year old version has lost
all meaning at this point.

changelog_begin
changelog_end
2021-06-22 15:11:47 +00:00
Moritz Kiefer
b98ad910fa
Bump test sha (#10055)
This needs bumping due to the off-ledger on-ledger speedy split.

changelog_begin
changelog_end
2021-06-18 10:03:55 +02:00
Remy
b8c02d65b7
Bump perf test for for #9871 (#9902)
CHANGELOG_BEGIN
CHANGELOG_END
2021-06-03 12:46:58 +00:00
Gary Verhaegen
83d60f0561
more frequent Windows cache cleanup (#9743)
We'd like this to run more than once an hour, and Azure doesn't support
that as a cron.

CHANGELOG_BEGIN
CHANGELOG_END
2021-05-19 14:24:47 +02:00
Moritz Kiefer
fb0551bdfd
Fix blackduck flag names (#9695)
changelog_begin
changelog_end
2021-05-16 16:45:31 +02:00
Andreas Herrmann
0acc4f115c
Patch old Bazel derivation (#9622)
The patch that it downloads via Nix was taken from a GH PR instead of a
commit such that the hash is not fully stable. This adds a patch to
download the relevant patch directly from a GitHub commit.

changelog_begin
changelog_end

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2021-05-07 18:23:47 +02:00
Stefano Baghino
3fa05804bf
Make non-repudiation integration exclusive to enterprise edition (#9219)
* Make non-repudiation integration exclusive to enterprise edition

Closes #9182

changelog_begin
changelog_end

* Address https://github.com/digital-asset/daml/pull/9219#discussion_r599578614

* Address https://github.com/digital-asset/daml/pull/9219#pullrequestreview-618605511

* Add aliases for CE test targets, thanks to @S11001001 for the feedback

* Uh, turns out you need a main...

* Address https://github.com/digital-asset/daml/pull/9219#discussion_r599717526

* Add missing dependency for Oracle
2021-03-24 20:25:47 +01:00
Gary Verhaegen
90aa84193f
tuesday notif: newlines (#9213)
CHANGELOG_BEGIN
CHANGELOG_END
2021-03-23 17:03:22 +01:00