The latest udpate of the BlackDuck script broke our run. In order to get
a successful scan tomorrow, this PR:
- Pins us back on yesterday's script version, and
- Adds an auto-update mechanism.
This way, we get to stay reasonably up-to-date with automated update
PRs, but we also get to choose when we upgrade.
* fix blackduck
BlackDuck is currently failing to run because it creates multiple report
files and then expects a single one. This is due to #16552 changing only
one of the output file names.
This PR resolves this by:
1. Changing the name everywhere, so all runs aggregate their results on
the same file.
2. Changing the PR step code to expect the given file name, rather than
try and glob it.
My understanding of the BlackDuck setup is very fuzzy so please
double-check carefully.
See #16600 for an example of what the update would be from this PR.
* Update daily-compat.yml
main instead of local-main-maven
---------
Co-authored-by: Brian Healey <brian.healey@digitalasset.com>
From output of currently failed jobs:
```
The key 'detect.npm.dependency.types.excluded' in property source
'commandLineArgs' contained a value that could not be reasonably
converted to the properties type. The exception was: Unable to parse raw
value 'NON_PRODUCTION' and coerce it into type 'NoneEnum or
NpmDependencyType'. Value was must be one of NONE,DEV,PEER
```
It looks like I was a bit overeager on #14995.
CHANGELOG_BEGIN
CHANGELOG_END
- The `check_releases` job seems to take between 4.5h and `TIMEOUT`
these days, with no clear cause for the large span in running time.
Hopefully bumping the limit from 4.5 to 8 hours will give us some
breathing room (this is not by any means a long-term solution, I'm well
aware of that).
- I couldn't find anybody who seemed to care about running the
create-daml-app compatibility tests on macOS, and they're super flaky
(typically requiring 5 reruns to succeed).
CHANGELOG_BEGIN
CHANGELOG_END
run-full-compat: true
After discussion with the team, we've reached the conclusion that the
JSON API is not changing enough for these performance tests to bring
much value: we're not in a situation where we think there's a high risk
of performance regressions, nor trying to focus on performance
improvements.
Note that this is only removing the tests from the daily run; the team
does see value in keeping the code for these tests around in case any
performance-focused work does arise in the future.
(This PR also disables running the `check_releases` job outside of main
branch runs, because there's really never been a good reason to.)
Fixes#14608.
CHANGELOG_BEGIN
CHANGELOG_END
run-full-compat: true
Since yesterday, this test is failing with an OOM error. I hoped I'd
have time to investigate today, but since I didn't and I'm off next week
disabling the test for now seems like the best option to let the overall
daily tests succeed.
The last known success was on 0bf28176a7,
and the first failure was on 444fff1fcf.
We don't test every commit, so any commit in-between could be the cause
of the issue.
Note that the failure does not seem to be flaky, but further testing may
be warranted.
CHANGELOG_BEGIN
CHANGELOG_END
There were two issues:
1. The `Directory.listFilesRecursive` function returns paths that
include its first argument, so there was never any overlap between
the sets.
2. That full path doesn't exist on AWS, and was given to AWS because
`s3Path opts f` returns just `f` if it starts with `/`.
This means that, on the one hand, the script tried to delete
_everything_ from the top-level, which is a bit bad, but, on the other
hand, the script did not actually manage to delete anything at all,
which is basically the status quo.
Note that deleting everything isn't _that_ bad as we reupload right
after and we have a cache in front on the site, so this would only
impact users if they were the first to try and view a given page in that
small time window between deletion and reupload.
This PR should fix both issues by removing the version-specific prefix.
CHANGELOG_BEGIN
CHANGELOG_END
There's a small, subtle bug that made its way into #14287: apparently
`/dev/stdout` is not defined on CI.
On the bright side, we don't actually need the `out` parameter, so
removing it seems like the easiest fix here.
CHANGELOG_BEGIN
CHANGELOG_END
* Merge ci/cron/wednesday.yml open_pr function into ci/bash-lib.yml
* remove unused function reset from ci/cron/wednesday.yml
* Request review from current release person for rotation PR
changelog_begin
changelog_end
The check_releases job has been a major player in the flakiness of the
daily test lately, simply by _timing out_ despite its 6h limit.
There are smarter, more "permanent" fixes we could implement here, but
as a quick stopgap measure I wanted to try out how much faster we would
go if we didn't need to reestablish a GCloud identity for each file.
CHANGELOG_BEGIN
CHANGELOG_END
run-full-compat: true
After yet another report of a ghost file, I stared at the code even
harder than before and finally figured out what was going wrong (or at
least one way it was going wrong): `listDirectory` is not recursive, so
the current code would only delete top-level files.
Hopefully this fixes that. I'll try to do some manual cleanup too.
CHANGELOG_BEGIN
CHANGELOG_END
With the new 2.0.0 release process, we're not releasing from the main CI
build anymore, so the compat update PRs are not getting created.
This restores that by making those PR creations part of the daily CI
build.
CHANGELOG_BEGIN
CHANGELOG_END
