We use `xargs` a lot but rely on the version on the user's system.
This adds it to dev-env so it's consistent everywhere.
If nothing else, this means we can safely use the `--no-run-if-empty`
flag, which is not available on BSD (and therefore macOS) `xargs`.
CHANGELOG_BEGIN
CHANGELOG_END
* dev-env: Add a symlink, `dev-env/jdk`, to the current JDK.
This makes configuring IntelliJ IDEA to use the correct JDK much easier.
CHANGELOG_BEGIN
CHANGELOG_END
* dev-env: Document setting up the JDK in IntelliJ IDEA.
daml-sdk-head: add optional sha information
This PR add an option, `--sha`, to `daml-sdk-head` so that it produces a
more accurate version number including the current git sha.
The main consequence is that calling `daml-sdk-head --sha` take
significantly longer than calling `daml-sdk-head`, because it needs to
recompile everything that depends on the version number.
> ## Wait, but why??
I started this work in support of an internal project that needed to
test against unreleased, and possibly unmerged, daml versions. However,
after further discussion I believe there is a better option for their
use-case. I've decided to still open this PR because the work was done
and there is no downside to it. It may still be useful if one wanted to
be able to maintain more than one non-released local version of daml.
CHANGELOG_BEGIN
CHANGELOG_END
There are two reasons for this:
1. 2018 was a long time ago and there seems to be no particular reason
for sticking to that version.
2. We have seen a bunch of issues where we get 404s when fetching msys
packages. The newer version has a larger mirrorlist which might help
with this. We could in principle try to patch the mirrorlist for
the old version but given that upgrading seems like a good idea
anyway, this seems easier.
changelog_begin
changelog_end
This does not get used very often so it is likely nobody will remember
how it works when we do use it. It's And due to the ordering Azure makes
of jobs in its UI, it's very easy to miss that there is a final,
Linux-based step and the values are actually printed there.
So this adds a little note to remind us of that.
Note that as this changes the `ci/patch_bazel_windows` folder, this will
also generate a new Bazel, so this PR will also update the Scoop
reference.
CHANGELOG_BEGIN
CHANGELOG_END
At the moment, with the `curl` call comfortably nested inside an `if`
statement, `curl` failures are interpreted as simply going down the
`else` branch of the conditional. Hoisting it up to a separate variable
declaration makes the script fail on the curl call itself.
CHANGELOG_BEGIN
CHANGELOG_END
It looks like the structure of the nix package has changed when we
updated nixpkgs in #6761, so we need to update the dev-env script to
match.
CHANGELOG_BEGIN
CHANGELOG_END
This file recently came up in a PR to fix shebang lines, and I realized
it's not been touched since open-sourcing and is not referenced
anywhere in the repo.
CHANGELOG_BEGIN
CHANGELOG_END
* Upgrade nixpkgs revision
* Remove unused minio
It used to be used as a gateway to push the Nix cache to GCS, but has
since been replaced by nix-store-gcs-proxy.
* Update Bazel on Windows
changelog_begin
changelog_end
* Fix hlint warnings
The nixpkgs update implied an hlint update which enabled new warnings.
* Fix "Error applying patch"
Since Bazel 2.2.0 the order of generating `WORKSPACE` and `BUILD` files
and applying patches has been reversed. The allows users to define
patches to these files that will not be immediately overwritten.
However, it also means that patches on another repository's original
`WORKSPACE` file will likely become invalid.
* a948eb7255
* https://github.com/bazelbuild/bazel/issues/10681
Hint: If you're generating a patch with `git` then you can use the
following command to exclude the `WORKSPACE` file.
```
git diff ':(exclude)WORKSPACE'
```
* Update rules_nixpkgs
* nixpkgs location expansion escaping
* Drop --noincompatible_windows_native_test_wrapper
* client_server_test using sh_inline_test
client_server_test used to produce an executable shell script in form of
a text file output. However, since the removal of
`--noincompatible_windows_native_test_wrapper` this no longer works on
Windows since `.sh` files are not directly executable on Windows.
This change fixes the issue by producing the script file in a dedicated
rule and then wrapping it in a `sh_test` rule which also works on
Windows.
* daml_test using sh_inline_test
* daml_doc_test using sh_inline_test
* _daml_validate_test using sh_inline_test
* damlc_compile_test using sh_inline_test
* client_server_test find .exe on Windows
* Bump Windows cache for Bazel update
Remove `clean --expunge` after merge.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Update rules_haskell hie-bios support
* Decouple Haskell ghcide and DAML ghcide
Creates a separate `stack_snapshot` to pull in `ghcide` for the Haskell
IDE use case independent of the `ghcide` for DAML. This allows to update
these two `ghcide` instances independently. As DAML uses `ghcide` the
library updates can be involved if the API experienced breaking changes.
At the same time we may wish to update `ghcide` for Haskell earlier to
make use of new features and stay compatible with rules_haskell's ghcide
support.
* Fix Haddock warnings reported by ghcide
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Update rules_haskell
* Pin stack_snapshot repositories
* Document stack_snapshot_json
CHANGELOG_BEGIN
CHANGELOG_END
* Don't pin stack_snapshot on Windows
The lock file is generated on Unix and includes unix specific
dependencies, e.g. `unix`. Most developers don't have easy access to a
Windows machine, so regenerating the lock file for Windows would be
inconvenient.
* upgrade stack 2.1.3 --> 2.3.1 on Windows
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
We don’t want to pass all flags as a single string. If we don’t have
any flags (which is the default) we end up passing an empty string
which breaks Bazel.
changelog_begin
changelog_end
* Use Distroless for the Java Docker base image.
We switched away from Distroless because it was causing issues with
`docker pull` when you had Docker configured to use `gcloud` for
authentication, but weren't actually authenticated.
Adding `docker-credential-gcloud` to dev-env should hopefully fix this,
meaning we can switch back to a base image that is better-maintained.
CHANGELOG_BEGIN
CHANGELOG_END
* Bump rules_docker to v0.14.3.
This fixes an issue when running `bazel sync`:
```
ERROR: java.io.IOException: Error downloading [http://central.maven.org/maven2/javax/servlet/javax.servlet-api/3.0.1/javax.servlet-api-3.0.1.jar] to [...]/external/javax_servlet_api/javax.servlet-api-3.0.1.jar: Unknown host: central.maven.org
```
It doesn't seem to be used anywhere. Obviously `git grep jo` returns a
lot of results, so I may have missed something in the noise, but I did a
reasonable effort to look through them.
CHANGELOG_BEGIN
CHANGELOG_END
Nix now requires -L, I’ve gone ahead and just normalized everything to
use -sfL which we were already using in one place.
changelog_begin
changelog_end
* Document a more modern way of using IntelliJ with our Bazel project.
Specifically, avoid `bazel-project-view` in favor of excluding a few
slow/unnecessary directories.
CHANGELOG_BEGIN
CHANGELOG_END
* Add .bazel-cache to .bazelignore instead of .bazelproject.
* Bazel: Don't mention a particular version of IntelliJ; just link.
Trust the canonical source.
The issues underlying #6173 prompted me to look into which `find`
version we had in dev-env, and I was surprised to notice we had none. We
already have `findutils` in our nix configuration, however, so this is
just adding the symlink.
Note that, on my machine at least, switching from the macOS-provided one
to the dev-env one does not change the result order for the hash
calculation in `ci/patch_bazel_windows`, so this would likely not have
helped for #6173. Still, we use `find` in many places in our scripts so
I think it's worth having there.
CHANGELOG_BEGIN
CHANGELOG_END
* Sort files when calculating CACHE_KEY
The order returned by `find` is unspecified and seems to have changed
for whatever reason in some cases. This changed the cache key which is
obviously not intended. It looks like the one we currently have in our
scoop manifest is the one that we get by sorting. Reversing the sort
produces the one CI currently calculates.
changelog_begin
changelog_end
* update manifest to match CI output
Co-authored-by: Gary Verhaegen <gary.verhaegen@digitalasset.com>
* Upgrade puppeteer
We’ve seen a couple of issues in the compatibility tests of the form
```
Error: Protocol error (Runtime.callFunctionOn): Target closed.
```
Looking at the issue tracker in puppeteer this might be fixed in newer
versions and I don’t see why we should stick to a fairly old version
anyway.
changelog_begin
changelog_end
* Upgrade nodejs
changelog_begin
changelog_end
* temporary add a step to kill node_modules
changelog_begin
changelog_end
* Kill live server and try to fix Windows
changelog_begin
changelog_end
* Undo rm
changelog_begin
changelog_end
I have never used Scoop before and I'm not sure how to actually test
this, but I thought this might at least get a conversation going.
Plus, if my reading of the documentation of both toxiproxy and Scoop is
correct, this may even work as is.
CHANGELOG_BEGIN
CHANGELOG_END
This PR removes a few scoop-related files that seem to have been there
primarily to help us discover how Scoop worked when we first started
using it. They have not been touched in a long time and as far as I can
tell have not been run in a long time either.
I'm a firm believer in the idea that dead code belongs to the git
history, and has no place in the current worktree.
CHANGELOG_BEGIN
CHANGELOG_END
* ADD: Change most Slack references to forum references where appropriate
CHANGELOG_BEGIN
Replace references to Slack with references to discuss.daml.com
CHANGELOG_END
* ADD: Change issue template support link from SO to forum
* ADD: Add back Slack links
It should not be an issue, but I know of at least 4 people for whom
having envsubst in dev-env causes their local git to get confused and
spit out many lines of warning on each invocation. This also seems to
make git much slower.
Given that we're only using envsubst in this one place, I think it may
be worth replacing.
CHANGELOG_BEGIN
CHANGELOG_END
* Include Bazel patch to cache exclusive tests on Windows
This includes the patch that we already use on Linux and MacOS to fix
caching of things marked exclusive. I’ve kept in the debugging output
that I added in the last patch for now. While our workaround seems to
be working, I’d like to wait a bit longer in case the issue reappears.
changelog_begin
changelog_end
* Actually bump manifest
changelog_begin
changelog_end
Bash is not easy to add to dev-env because dev-env depends on Bash. This
has not been an issue so far because Bash behaves in very sensible ways
overall and is mostly backwards compatible.
A recent change to `daml-sdk-head` is, however, using some feature of
Bash 4 that is not evailable in Bash 3. Bash 3 is ancient so in an ideal
world that would not be an issue, but macOS still ships with Bash 3 for
some obscure (licensing) reason.
It turns our that the line
```bash
arr=()
```
in Bash 4 sets the variable `arr` to an empty array, whereas it leaves it
unset in Bash 3. This means that the later use of `arr`, in the case
where no further element has been added to the array, will yield an
error in combination with the `set -u` option we are using.
This PR changes the usage pattern from
```bash
"${arr[@]}"
```
which fails on Bash 3 to
```bash
${arr[@]:-}
```
which works as expected. Note that the quotes have been removed: the
quotes in this case are not useful assuming that the flags themselves
are never multiword. Without the quotes, this evaluates to `""` under
Bash 3 (as the variable is not set), which confuses Bazel because now it
thinks it's asked to build the `""` target, which it has no rule for.
Ignoring the quoting issue, the actual fix is to include `:-` inside the
`{}`, which instructs Bash to replace the variable with what follows the
`-` in case the variable is not set (in this case, an empty string), and
therefore not crash on this specific variable not being set despite the
`set -u` option.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-sdk-head: Quote all variables.
And use arrays where necessary.
* daml-sdk-head: Use Bash tests (`[[` and `]]`) rather than `test`.
Most importantly, they support the `&&` and `||` operators, because
they're a shell builtin, not a program.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-sdk-head: Refer to `${BASH_SOURCE[0]}` explicitly.
`BASH_SOURCE` is an array. Expanding an array gets the first element, but this
is unclear. We can clarify by explicitly expanding the first element.
* daml-sdk-head: Use `command -v` instead of `which`, as it's more standard.
* daml-sdk-head: Use semicolons judiciously.
* Add debugging output to inclusion errors
This adds some more debugging output to inclusion errors in Bazel
which should hopefully help us track it down. These are the only call sites
in the Bazel source that can produce them. My suspicion is that it’s
coming from HeaderDiscovery but I’m not entirely sure what is off.
We’ll almost certainly have to add more output once we know which of
those 3 cases we hit but let’s do it step by step.
changelog_begin
changelog_end
* Bump url and hash
changelog_begin
changelog_end
patch Bazel on Windows (ci setup)
We have a weird, intermittent bug on Windows where Bazel gets into a
broken state. To investigate, we need to patch Bazel to add more debug
output than present in the official distribution. This PR adds the basic
infrastructure we need to download the Bazel source code, apply a patch,
compile it, and make that binary available to the rest of the build.
This is for Windows only as we already have the ability to do similar
things on Linux and macOS through Nix.
This PR does not contain any intresting patch to Bazel, just the minimum
that we can check we are actually using the patched version.
CHANGELOG_BEGIN
CHANGELOG_END
This PR adds a simple daily job that runs the performance test on a
chosen "baseline" commit and then runs the same benchmark on latest
master. This should allow us to track overall performance improvements.
CHANGELOG_BEGIN
CHANGELOG_END
The python install step started failing because there is a new pip
release so we get a warning. I tried to just upgrade python but the
new python manifest depends on functions that need an upgrade of scoop
so I upgraded that as well. Not quite sure which of those upgrades
fixes the issue …
changelog_begin
changelog_end
* Use com.daml as groupId for all artifacts
CHANGELOG_BEGIN
[SDK] Changed the groupId for Maven artifacts to ``com.daml``.
CHANGELOG_END
* Add 2 additional maven related checks to the release binary
1. Check that all maven upload artifacts use com.daml as the groupId
2. Check that all maven upload artifacts have a unique artifactId
* Address @cocreature's comments in https://github.com/digital-asset/daml/pull/5272#pullrequestreview-385026181
* Bump openssl
The previous one has stopped working for some reason :sadpanda:
changelog_begin
changelog_end
* daml-assistant: Add `--wall-clock-time` to the Sandbox Next test.
Missed this due to doing two things at once.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Update rules_haskell
The workaround for linking against `Cffi` in the REPL has been
upstreamed in a more generalized form.
CHANGELOG_BEGIN
CHANGELOG_END
* ghcide: Use rules_haskell's hie-bios support
* Document `ghcide` Bazel integration
* Rename files to match module names
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
