* Try to upgrade protobuf docs plugin
changelog_begin
changelog_end
* Fix extension number 65020 is already registered
Building `//ledger-api/grpc-definitions:ledger-api-docs` [failed
with](https://github.com/digital-asset/daml/issues/11761#issuecomment-978947565)
```
panic: proto: extension number 65020 is already registered on message google.protobuf.FieldOptions
```
Go dependencies are now pulled in via Gazelle. By default Gazelle will
generate new proto rules for any `.proto` files encountered in third
party Go dependencies. However, many of these already have pregenerated
`.pb.go` files generate with the appropriate configuration.
The problem can be avoided by configuring Gazelle to not generate new
proto rules, but instead use pre-existing `.pb.go` files.
For reference the field number is set in
[go-proto-validators](32a686adf8/validator.proto (L19))
which is an indirect dependency through protoc-gen-doc.
In this case we need to update protoc-gen-validate to v0.6.2 to include
4f41f10dde
which fixes unknown label errors.
* ./fmt
* Expose gRPC status.proto for Haskell bindings
* Update Gazelle to support embedsrcs on Windows
`protoc-gen-doc` relies on `go:embed` file embedding
2dde01902b/resources.go (L8).
Gazelle supports `embedsrcs`, however, it did not generate the attribute
correctly on Windows due to the different directory separator. This is
fixed in https://github.com/bazelbuild/bazel-gazelle/pull/1101.
* Add gazelle to compatibility workspace
It's loaded into `@daml`'s top-level `BUILD` file and ends up being a
dependency of the compatibility workspace as well.
* shift go_googleapis import
* Delete dead code
protobuf is imported transitively.
* Document how to add Go dependencies
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Update to Java 11
changelog_begin
changelog_end
* Fix RoundingMode deprecation warnings
* Fix dep-ann warning
* Integer constructor
* JavaX annotation dependency
* javax.xml.bind was removed in Java 11
Using Guava as a replacement, since it is already a project dependency.
* JDK 11 no longer has a separate JRE tree
* Remove unused jdk_nix import
* remove now redundant jdk11_nix
* Java 8 --> 9 increased Instant.now() precision
See https://bugs.openjdk.java.net/browse/JDK-8068730
The precision of `Instant.now()` increased between Java 8 and Java 9.
On Linux and MacOS this doesn't seem to be a problem, as the precision
still seems to be at micro seconds. However, on Windows this now causes
errors of the following form:
```
java.lang.IllegalArgumentException: Conversion of Instant
2021-11-05T13:58:56.726875100Z to microsecond granularity would result
in loss of precision.
```
Suggesting that it now offers sub-microsecond precision.
`TimestampConversion.instantToMicros` had a check to fail if the
conversion lead to a loss of precision. In the specific failing test
case this is not a concern, so this adds a `roundInstantToMicros`
variant that avoids this kind of error.
* TMP round timestamps
* Revert "TMP round timestamps"
This reverts commit af8e261278.
* Skip versions before 1.6.0 in migration tests
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Fix rules_scala Windows launcher
* Define nixpkgs_java_configure
To generate a java_runtime based on a Nix provided JDK similar to how it
is done with the Python and CC toolchain.
changelog_begin
changelog_end
* Enable Nix provided Java runtime on Linux and MacOS
* Define dadew_java_configure
To import a JDK and JAVA_HOME from dadew similar to
nixpkgs_java_configure.
* Enable dadew Java runtime on Windows
* Remove now unused java_home_runtime
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* compatibility: Use the same Scala version as the root.
It's confusing to get out of sync. `compatibility` was using Scala
2.12.12 while the root was on 2.12.13.
This brings them in sync to 2.12.13.
CHANGELOG_BEGIN
CHANGELOG_END
* Factor out Scala artifacts into scala_version.bzl.
* Bazel: Explicitly specify artifact hashes for Scala 2.13.
Not strictly necessary, as we _happen_ to depend on the same point
release as the Bazel Scala plugin, but this is incidental and we don't
want to bank on it.
* Upgrade Scala 2.12 to v2.12.13.
This is being pulled in anyway because of Maven/Gradle/etc's fun
"favor the most recent" resolution mechanism. The version of Akka we
depend upon transitively depends on Scala 2.12.13, so any downstream
consumers will see that as the Scala version required.
Bringing the Daml repo in line means no more confusion.
CHANGELOG_BEGIN
- [Scala Bindings] If you are using Daml on Scala 2.12, it now depends
on Scala v2.12.13 (from v2.12.12).
CHANGELOG_END
* Scala 2.12.13 is the default version in our pinned version of nixpkgs.
* Upgrade Scala 2.13's Wartremover version.
* Rename `scala_version_rule` to `scala_version_configure`.
* Add a test case to ensure the Scala versions are the same everywhere.
* Add tests for the Scala JAR versions in maven_install_*.json
* gatling-utils: Change the sort order of the expected CSV in tests.
I don't know why this changed, but it seems to be stable.
* compatibility: `scala_version` -> `scala_version_configure`.
* Bazel: Disable the Scala version tests on Windows.
* compatibility: Upgrade Wartremover to Scala 2.12.13.
* Fix compat job
changelog_begin
changelog_end
* Cleanup head_sdk
changelog_begin
changelog_end
* turns out removing everything also removes files that shouldn’t be removed
changelog_begin
changelog_end
* gnah
changelog_begin
changelog_end
* maybe this was a bad idea
changelog_begin
changelog_end
* trynottocry
changelog_begin
changelog_end
* Add a Scala 2.13 build pipeline
This adds initial support for multiple Scala versions controlled via
the DAML_SCALA_VERSION env var and a CI job to make sure we don’t
regress. For now we only test //libs-scala/ports/... which seemed like
the easiest starting point I could find. We can incrementally expand
that over time.
changelog_begin
changelog_end
* Document pinning
changelog_begin
changelog_end
* Address review comments
changelog_begin
changelog_end
This is necessary to at least attempt an upgrade to 2.13 and
generally, I want to keep our rulesets up2date. rules-scala forces the
version of scalatest so we have to bump that at the same time.
This requires changes to basically all Scala test suites since the
import structure has changed and a bunch of things (primarily
scalacheck support) got split out.
Apologies for the giant PR, I don’t see a way to keep it smaller.
changelog_begin
changelog_end
* Update rules_haskell
Removes the warning about Bazel 3.3.1 being too recent.
* Remove unused rules_haskell patch
changelog_begin
changelog_end
* fmt
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Factor out tar/gzip reproducibility flags
* use mktgz in package-app
* Bazel managed tar/gzip
* Remove quiet = True
As stated in the comment this is no longer required with Bazel >= 3.0.
* Build package-app as a sh_binary
This way Bazel will manage the runtime dependencies tar, gzip, mktgz,
and patchelf.
package-app.sh changes directory so it needs to make sure that all paths
are absolute and that the runfiles tree/manifest location is forwarded
to programs called by package-app.sh.
* Avoid file path too long errors
* Fix readlink -f on MacOS
* Document abspath
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* upgrade to wartremover 2.4.9
* simplify wart list and list JavaConversions as disabled
* no changelog
CHANGELOG_BEGIN
CHANGELOG_END
* delete long-obsolete, contradictory comment
* also upgrade wartremover in compatibility (leaving aside maven_install.json)
* update compatibility maven_install.json to match
This PR does two things:
1. It makes use of the new sha256sums file to speed up the script to
update the versions significantly. The code here is a bit hacky but
as mentioned in the comment, this script is only run offline atm so
that seems reasonable.
2. It prunes SDK snapshot versions to those for which there is no
corresponding stable release.
changelog_begin
changelog_end
* Upgrade puppeteer
We’ve seen a couple of issues in the compatibility tests of the form
```
Error: Protocol error (Runtime.callFunctionOn): Target closed.
```
Looking at the issue tracker in puppeteer this might be fixed in newer
versions and I don’t see why we should stick to a fairly old version
anyway.
changelog_begin
changelog_end
* Upgrade nodejs
changelog_begin
changelog_end
* temporary add a step to kill node_modules
changelog_begin
changelog_end
* Kill live server and try to fix Windows
changelog_begin
changelog_end
* Undo rm
changelog_begin
changelog_end
This is preparatory work for hitting the transaction service (and
others but that one is the most important) which isn’t possible via
DAML Script. To ease review, this PR only switches from a DAML Script
to a custom Scala binary. It does not yet change what is tested.
changelog_begin
changelog_end
* update rules_nixpkgs
* Use hermetic nixpkgs cc toolchain
CHANGELOG_BEGIN
CHANGELOG_END
* Work around Bazel's cc toolchain autodetection
* Use --crosstool_top for hermetic cc toolchain
When using --incompatible_enable_cc_toolchain_resolution instead
cc actions still depend on
`external/local_config_cc/builtin_include_directory_paths`
as well as
`external/nixpkgs_cc_toolchain_config/builtin_include_directory_paths`.
* override local_config_cc
* remove unused attribute
* Fix posix toolchain on Windows
* nixpkgs cc toolchain not on Windows
* Fix nixpkgs cc toolchain on MacOS
* nixpkgs cc toolchain uses bin/cc
* Use darwin.binutils on MacOS
* Remove clang(++) and gcc (g++) symlinks
The toolchain only considers `bin/cc` and having the other symlinks
around could lead to confusion
* Use hermetic toolchain in compatibility workspace
* Avoid empty linker flags
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Include the first feature in compat tests for create-daml-app
This adds the step from the create-daml-app tests that applies the
patch for the messaging feature and tests that things will build. This
is the last step before we can actually run the puppeteer tests which
will turn this into an actual compatibility test.
changelog_begin
changelog_end
* temporarily run all tests
changelog_begin
changelog_end
* Fix exclusions
changelog_begin
changelog_end
* Revert "temporarily run all tests"
This reverts commit 7425dd09cf48f2bfd6064b55d857c76d51afc821.
* Remove accidentally committed messaging.patch
changelog_begin
changelog_end
This is the first part of #5700
It adds tests that build create-daml-app using `daml build` and then
run the codegen and build the UI. Contrary to our main tests these
also run on Windows. This is actually reasonably simple by first
building the typescript libraries on Linux and then downloading them
on Windows.
There are two parts that are still missing from the tests in the main
workspace:
1. Building the extra feature. This should be fairly easy to add.
2. Running the pupeeter tests. At least MacOS and Linux should be
reasonably easy. I don’t know what horrors Windows will throw at
us. This step is what actually makes this a compatibility
test. Currently it doesn’t actually launch Sandbox and the JSON API.
Since this PR is already pretty large, I’d like to tackle those things
separately.
changelog_begin
changelog_end
* Update SDK versions in compatibility tests
This adds a Haskell script to generate a versions.bzl file that
contains the list of versions as well as their hashes. This should
make it a bit easier to keep things up2date going forward.
The script is a bit slow since downloading all the SDKs takes quite a
while but for now it should be good enough and is much more pleasant
than having to figure this out manually.
changelog_begin
changelog_end
* Address review comments
changelog_begin
changelog_end
* Fix excluded tests
changelog_begin
changelog_end
* Make compat tests work on windows
This required some changes to the daml_sdk rule since the read-only
installation by the assistant breaks Bazel completely. We could only
apply those changes on Windows but I think I prefer the consistency
across platforms here over trying to stay close to how the SDK is
installed on user machines given that the SDK installation is not
something we’ve had issues with.
I’ve excluded the postgresql tests for now. I don’t expect them to be
particularly hard to fix but I’ve already spent almost 2 days on this
and having some tests run on Windows seems like a clear improvement
over running no tests on Windows :)
changelog_begin
changelog_end
* Remove todo
changelog_begin
changelog_end
* Copy daml ledger test to compatibility
Specifically:
- `daml-assistant/daml-helper/test/DA/Daml/Helper/Test/Deployment.hs`
- `libs-haskell/test-utils/DA/Test/Sandbox.hs`
* Adapt daml ledger test for compatibility
Avoid dependencies on the daml repository and make the SDK and platform
components configurable.
* Add test certificates to compatibility
Taken from `@daml//ledger/test-common/test-certificates`.
* Define daml_ledger_test
CHANGELOG_BEGIN
CHANGELOG_END
* Add note on keeping daml ledger tests in sync
* ./fmt.sh
* Strictly require --sdk-version flag
Tasty doesn't allow options without a default value, so we default to
`error` to enforce a value being set.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
This PR extends the existing Linux compatibility tests to run on macOS
too. Fixes#5692.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Fix ./fmt.sh check
We accidentally introduced hlint lints when we added the compatibility
workspace. The hlint code in ./fmt.sh correctly detects that there is
a hint but then exits with 0 since it uses the exit code from `echo`
rather than the one from `hlint`. The info that we printed with
`echo` didn’t actually make any sense so I added a new comment as to
why we run hlint here. This requires some setup changes so that the
compatibility workspace gets the same Haskell flags since otherwise
using the same hlint rules obviously does not make sense.
Fixes#5701
changelog_begin
changelog_end
* Update fmt.sh
Co-Authored-By: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Extend compatibility tests to DAML on SQL
This feels a bit hacky since the runfiles don’t work quite like I
would expect them to but it’s at least not more hacky than what we do
for the head-based tests we currently have.
Progress towards #5695
changelog_begin
changelog_end
* Fix runfiles with more bash
changelog_begin
changelog_end
* remove redundant port options
changelog_begin
changelog_end
* Create fewer sandbox targets
changelog_begin
changelog_end
* Apply suggestions from code review
Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* Fix runfiles snippet
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
This is a first step towards testing cross-version
compatibility. It doesn’t actuall do much yet but hopefully it should
be easier to parallelize once we have the initial boilerplate in place
so ideally I’d like to address most missing things and issues in
separate PRs.
changelog_begin
changelog_end
