* Use nix-provided gRPC on Unix-like OSs
This fixes the da-ghcid command on macOS and Linux.
Unfortunately on macos at least on CI it still seems to segfault when
trying to run damlc in ghci. However, at least typechecking works
so this is clearly progress.
changelog_begin
changelog_end
* Test case for LockedFreePort not colliding with port 0
changelog_begin
changelog_end
* Discover dynamic port range on Linux
* Random port generator outside ephemeral range
* remove dev comments
* Draw FreePort from outside the ephemeral port range
Note, there is a race condition between the socket being closed and the
lock-file being created in LockedFreePort. This is not a new issue, it
was already present with the previous port 0 based implementation.
LockedFreePort handles this by attempting to find a free port and taking
a file lock multiple times.
But, it could happen that A `find`s port N, and obtains the lock, but
doesn't bind port N again, yet; then B binds port N during `find`; then
A attempts to bind port N before B could release it again and fails
because B still holds it.
* Select dynamic port range based on OS
* Detect dynamic port range on MacOS and Windows
* Import sysctl from Nix on MacOS
changelog_begin
changelog_end
* Windows line separator
* FreePort helpers visibility
* Use more informative exception types
* Use a more light weight unit test
* Add comments
* Fix Windows
* Update libs-scala/ports/src/main/scala/com/digitalasset/ports/FreePort.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update libs-scala/ports/src/main/scala/com/digitalasset/ports/FreePort.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Add a comment to clarify the generated port range
* fmt
* unused import
* Split libs-scala/ports
Splits the FreePort and LockedFreePort components into a separate
library as this is only used for testing purposes.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Upgrade protobuf-java and scalapb.
CHANGELOG_BEGIN
CHANGELOG_END
* Bazel: Generate ScalaPB classes without `unknownFields`.
This is very annoying because it means that extractors need an extra
`_` for no good reason. It also breaks Circe's automatic derivation.
Including unknown fields is the default behavior in Protocol Buffers for
Java now, and we'll probably have to reckon with it at some point, but
let's kick that can down the road a little.
* Upgrade protobuf-java to 3.17.1.
This is identical to 3.17.0, but let's track it anyway.
* Bazel: Move ScalaPB versions into their own file.
They don't need to go into the generated one.
* participant-integration-api: Build Oracle tests, but don't run them.
CHANGELOG_BEGIN
CHANGELOG_END
* triggers: Switch to an environment variable for enabling Oracle tests.
* http-json: Switch to an environment variable for enabling Oracle tests.
* Disable running Oracle tests by default, not building them.
* triggers/service: Remove unused test dependencies.
* compatibility: Use the same Scala version as the root.
It's confusing to get out of sync. `compatibility` was using Scala
2.12.12 while the root was on 2.12.13.
This brings them in sync to 2.12.13.
CHANGELOG_BEGIN
CHANGELOG_END
* Factor out Scala artifacts into scala_version.bzl.
* Bazel: Explicitly specify artifact hashes for Scala 2.13.
Not strictly necessary, as we _happen_ to depend on the same point
release as the Bazel Scala plugin, but this is incidental and we don't
want to bank on it.
* Upgrade Scala 2.12 to v2.12.13.
This is being pulled in anyway because of Maven/Gradle/etc's fun
"favor the most recent" resolution mechanism. The version of Akka we
depend upon transitively depends on Scala 2.12.13, so any downstream
consumers will see that as the Scala version required.
Bringing the Daml repo in line means no more confusion.
CHANGELOG_BEGIN
- [Scala Bindings] If you are using Daml on Scala 2.12, it now depends
on Scala v2.12.13 (from v2.12.12).
CHANGELOG_END
* Scala 2.12.13 is the default version in our pinned version of nixpkgs.
* Upgrade Scala 2.13's Wartremover version.
* Rename `scala_version_rule` to `scala_version_configure`.
* Add a test case to ensure the Scala versions are the same everywhere.
* Add tests for the Scala JAR versions in maven_install_*.json
* gatling-utils: Change the sort order of the expected CSV in tests.
I don't know why this changed, but it seems to be stable.
* compatibility: `scala_version` -> `scala_version_configure`.
* Bazel: Disable the Scala version tests on Windows.
* compatibility: Upgrade Wartremover to Scala 2.12.13.
This fixes Scaladoc and our pom file generation.
It also clears up the confusing error around gatling and removes a
redundant dependency on sbt (no idea why we had that in the first
place) both of which resulted in Scala 2.12 dependencies in our 2.13
lockfile which is obviously bad.
With this, we should now be ready to publish Scala 2.13 artifacts once
the ledger API test tool PR lands.
changelog_begin
changelog_end
* Clean broken entries from the Bazel cache
This is hopefully a somewhat reasonable workaround for the "output not
created" errors that keep annoying us.
For now, this is just part of the hourly cronjob but we could move it
somewhere else if desired.
changelog_begin
changelog_end
* Fix GCS credentials
changelog_begin
changelog_end
* Add a Scala 2.13 build pipeline
This adds initial support for multiple Scala versions controlled via
the DAML_SCALA_VERSION env var and a CI job to make sure we don’t
regress. For now we only test //libs-scala/ports/... which seemed like
the easiest starting point I could find. We can incrementally expand
that over time.
changelog_begin
changelog_end
* Document pinning
changelog_begin
changelog_end
* Address review comments
changelog_begin
changelog_end
* Bump rules_nodejs
rules_nodejs finally added support for module remapping via js_library
so we can kill our custom rule for that.
changelog_begin
changelog_end
* Fix windows
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
This is necessary to at least attempt an upgrade to 2.13 and
generally, I want to keep our rulesets up2date. rules-scala forces the
version of scalatest so we have to bump that at the same time.
This requires changes to basically all Scala test suites since the
import structure has changed and a bunch of things (primarily
scalacheck support) got split out.
Apologies for the giant PR, I don’t see a way to keep it smaller.
changelog_begin
changelog_end
* Upgrade ledger-api-test-tool-on-canton to the canton 0.19.0 release
- Reflecting changes related to the Community edition of canton
- Testing full ContractKeysIT in a separate canton environment with
unique contract keys. The ability to run multiple canton environments
will come in handy when we introduce the participant pruning test suite.
- Slight canton configuration changes
CHANGELOG_BEGIN
CHANGELOG_END
* Formatting
* Review feedback by Samir
* Fixup
* Enables fetching scala library sources
Enables fetch_sources flag for fetching scala sources.
These sources can be used for manual linking scala library if sources are not resolved otherwise
CHANGELOG_BEGIN
CHANGELOG_END
* Apply suggestions from code review
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Remove authentication service from trigger service
This rips out the integration with ref-ledger-authenticator and
together with that the weird http basic auth which doesn’t actually
work for ref-ledger-authenticator outside of its testing mode and also
includes some very sketchy stuff like storing user
passwords (encrypted but still).
While the exact details of the new approach towards auth are still not
completely clear, it is clear that it is going to be sufficiently
different from what we did before that starting from a clean slate is
easier.
changelog_begin
- [Trigger Service] Parties are now specified in request bodies as
opposed to via HTTP Basic auth. This is done in preparation for
running the trigger service against authenticated ledgers.
changelog_end
* Fix integration tests
changelog_begin
changelog_end
* Update rules_haskell
Removes the warning about Bazel 3.3.1 being too recent.
* Remove unused rules_haskell patch
changelog_begin
changelog_end
* fmt
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Factor out tar/gzip reproducibility flags
* use mktgz in package-app
* Bazel managed tar/gzip
* Remove quiet = True
As stated in the comment this is no longer required with Bazel >= 3.0.
* Build package-app as a sh_binary
This way Bazel will manage the runtime dependencies tar, gzip, mktgz,
and patchelf.
package-app.sh changes directory so it needs to make sure that all paths
are absolute and that the runfiles tree/manifest location is forwarded
to programs called by package-app.sh.
* Avoid file path too long errors
* Fix readlink -f on MacOS
* Document abspath
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Use Bazel builtin pkg_tar rule
* Use @rules_pkg//:pkg.bzl%pkg_tar
The pkg_tar rule builtin to Bazel has been deprecated.
See https://docs.bazel.build/versions/master/be/pkg.html
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* daml-on-sql + sandbox: Remove unused Bazel `load` declarations.
* daml-on-sql: Build a Docker image.
* daml-on-sql: Move the example Kubernetes file over from Sandbox.
* Add a changelog entry that was dropped.
CHANGELOG_BEGIN
* [DAML on SQL] DAML on SQL is now available with every release,
from the GitHub Releases page.
CHANGELOG_END
* Upgrade the Java Docker base image to the latest version.
* WIP Call authorize before start trigger
* Intro auth service client
* Auth service client test setup
CHANGELOG_BEGIN
CHANGELOG_END
* scalafmt
* Use Spray JSON format and slightly better uri handling
* Get auth client test to work and fiddle with uris
* Remove changes to Server
* Add package for URI
* Format
* Copyrights
* Update sandbox packages
* Use default wallclock time for auth service sandbox
* Use Distroless for the Java Docker base image.
We switched away from Distroless because it was causing issues with
`docker pull` when you had Docker configured to use `gcloud` for
authentication, but weren't actually authenticated.
Adding `docker-credential-gcloud` to dev-env should hopefully fix this,
meaning we can switch back to a base image that is better-maintained.
CHANGELOG_BEGIN
CHANGELOG_END
* Bump rules_docker to v0.14.3.
This fixes an issue when running `bazel sync`:
```
ERROR: java.io.IOException: Error downloading [http://central.maven.org/maven2/javax/servlet/javax.servlet-api/3.0.1/javax.servlet-api-3.0.1.jar] to [...]/external/javax_servlet_api/javax.servlet-api-3.0.1.jar: Unknown host: central.maven.org
```
* Copy code button
Added the sphinx copy code extension (https://sphinx-copybutton.readthedocs.io/en/latest/) to the docs. Tested with and it worked.
CHANGELOG_START
CHANGELOG_END
* Remove vendored copy of sphinx_copybutton
changelog_begin
changelog_end
* remove debugging output
changelog_begin
changelog_end
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
Canton test suite had been disabled shortly before DAML 1.0. Reenabling
those tests that have been running flake-free in Canton for over a month.
In addition:
- Switched test selection from --include to --exclude so that we see how newly
added suites run multi-node.
- Switched Canton from in-memory to h2. Also adopted other config settings in
canton.conf from how tests are invoked in canton continuous integration.
- Increased number of participants from 2 to 4.
- Upgraded Canton from 0.11.0 to 0.15.0
CHANGELOG_BEGIN
CHANGELOG_END
* Upgrade puppeteer
We’ve seen a couple of issues in the compatibility tests of the form
```
Error: Protocol error (Runtime.callFunctionOn): Target closed.
```
Looking at the issue tracker in puppeteer this might be fixed in newer
versions and I don’t see why we should stick to a fairly old version
anyway.
changelog_begin
changelog_end
* Upgrade nodejs
changelog_begin
changelog_end
* temporary add a step to kill node_modules
changelog_begin
changelog_end
* Kill live server and try to fix Windows
changelog_begin
changelog_end
* Undo rm
changelog_begin
changelog_end
* io.grpc:grpc-xxxx to 1.29.0 (from 1.22.1)
io.netty:netty-xxxx to .1.50.Final (from 4.1.37.Final)
io.nett.netty-tcp-native-boringssl-static to 2.0.30.Final (from 2.0.25.Final)
To resolve open vulnerabilities with these versions
netty-4.1.37.Final vulnerabilities
BDSA-2018-4022 (Medium)
BDSA-2019-2610 (Medium)
BDSA-2019-3119 (CVE-2019-16869) (Medium)
BDSA-2020-0130 (Medium)
BDSA BDSA-2019-4230 (CVE-2019-20445) (Low)
BDSA BDSA-2019-4231 (CVE-2019-20444) (Low)
BDSA BDSA-2020-0666 (CVE-2020-11612) (Low)
BDSA BDSA-2019-2642 (Low)
BDSA BDSA-2019-2649 (Low)
BDSA BDSA-2019-2643 (Low)
CHANGELOG_BEGIN
Upgrade io.grpc:grpc-xxxxx and io.netty:netty-xxx version to latest
released to avoid exposure to reported security vulnerabilities in
currently used versions
CHANGELOG_END
–
* Update spray versions to address vulnerabilities
CVE-2018-18853 and CVE-2018-18854
CHANGELOG_BEGIN
Upgrade io.grpc:grpc-xxxxx and io.netty:netty-xxx version to latest
released to avoid exposure to reported security vulnerabilities in
currently used versions
CHANGELOG_END
* do not change io.grpc version since reflection seems to be misbehaving
* Clarify how to bump grpc/netty/protobuf versions
Also "downgrade" netty to 4.1.48, according to
https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty
CHANGELOG_BEGIN
CHANGELOG_END
* Load protobuf deps after haskell deps to avoid loading an older version of rules_cc
* Upgrade protoc and protobuf-java to 3.11.0
* buildifier reformat
* regen unique int after rebase
* remove commented patch
Co-authored-by: Gerolf Seitz <gerolf.seitz@daml.com>
The comment on it says it should be replaced by `local_jdk//:javadoc`
once [a patch](https://github.com/bazelbuild/bazel/pull/7898) is merged
into Bazel. Not only has that patch been merged, but I could not find
any reference to `javadoc_dev_env` to replace.
CHANGELOG_BEGIN
CHANGELOG_END
First version of static verification tool.
The current state of the tool:
- Reads DAR files.
- Partially evaluates the code.
- Generates constraints for the field and choice to be verified.
- Passes the constraints to an SMT solver.
- Some basic tests.
* update rules_nixpkgs
* Use hermetic nixpkgs cc toolchain
CHANGELOG_BEGIN
CHANGELOG_END
* Work around Bazel's cc toolchain autodetection
* Use --crosstool_top for hermetic cc toolchain
When using --incompatible_enable_cc_toolchain_resolution instead
cc actions still depend on
`external/local_config_cc/builtin_include_directory_paths`
as well as
`external/nixpkgs_cc_toolchain_config/builtin_include_directory_paths`.
* override local_config_cc
* remove unused attribute
* Fix posix toolchain on Windows
* nixpkgs cc toolchain not on Windows
* Fix nixpkgs cc toolchain on MacOS
* nixpkgs cc toolchain uses bin/cc
* Use darwin.binutils on MacOS
* Remove clang(++) and gcc (g++) symlinks
The toolchain only considers `bin/cc` and having the other symlinks
around could lead to confusion
* Use hermetic toolchain in compatibility workspace
* Avoid empty linker flags
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
