* Update rules_haskell
Removes the warning about Bazel 3.3.1 being too recent.
* Remove unused rules_haskell patch
changelog_begin
changelog_end
* fmt
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Use Bazel builtin pkg_tar rule
* Use @rules_pkg//:pkg.bzl%pkg_tar
The pkg_tar rule builtin to Bazel has been deprecated.
See https://docs.bazel.build/versions/master/be/pkg.html
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
I also switched zlib from a random looking commit to the corresponding
tag (which is the latest release) which points to the same commit.
changelog_begin
changelog_end
This does not change the version of rules_apple, it only pins the http
archive instead of fetching via git tag.
To avoid Bazel warnings of the following form since Bazel 3.3.1
```
DEBUG: Rule 'build_bazel_rules_apple' indicated that a canonical reproducible form can be obtained by modifying arguments commit = "ff6a37b24fcbbd525a5bf61692a12c810d0ee3c1", shallow_since = "1559833568 -0700" and dropping ["tag"]
DEBUG: Repository build_bazel_rules_apple instantiated at:
no stack (--record_rule_instantiation_callstack not enabled)
Repository rule git_repository defined at:
/home/aj/.cache/bazel/_bazel_aj/f66bee630c6a2cd906f92a0f5cdf8769/external/bazel_tools/tools/build_defs/repo/git.bzl:195:33: in <toplevel>
```
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Upgrade nixpkgs revision
* Remove unused minio
It used to be used as a gateway to push the Nix cache to GCS, but has
since been replaced by nix-store-gcs-proxy.
* Update Bazel on Windows
changelog_begin
changelog_end
* Fix hlint warnings
The nixpkgs update implied an hlint update which enabled new warnings.
* Fix "Error applying patch"
Since Bazel 2.2.0 the order of generating `WORKSPACE` and `BUILD` files
and applying patches has been reversed. The allows users to define
patches to these files that will not be immediately overwritten.
However, it also means that patches on another repository's original
`WORKSPACE` file will likely become invalid.
* a948eb7255
* https://github.com/bazelbuild/bazel/issues/10681
Hint: If you're generating a patch with `git` then you can use the
following command to exclude the `WORKSPACE` file.
```
git diff ':(exclude)WORKSPACE'
```
* Update rules_nixpkgs
* nixpkgs location expansion escaping
* Drop --noincompatible_windows_native_test_wrapper
* client_server_test using sh_inline_test
client_server_test used to produce an executable shell script in form of
a text file output. However, since the removal of
`--noincompatible_windows_native_test_wrapper` this no longer works on
Windows since `.sh` files are not directly executable on Windows.
This change fixes the issue by producing the script file in a dedicated
rule and then wrapping it in a `sh_test` rule which also works on
Windows.
* daml_test using sh_inline_test
* daml_doc_test using sh_inline_test
* _daml_validate_test using sh_inline_test
* damlc_compile_test using sh_inline_test
* client_server_test find .exe on Windows
* Bump Windows cache for Bazel update
Remove `clean --expunge` after merge.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Update rules_haskell hie-bios support
* Decouple Haskell ghcide and DAML ghcide
Creates a separate `stack_snapshot` to pull in `ghcide` for the Haskell
IDE use case independent of the `ghcide` for DAML. This allows to update
these two `ghcide` instances independently. As DAML uses `ghcide` the
library updates can be involved if the API experienced breaking changes.
At the same time we may wish to update `ghcide` for Haskell earlier to
make use of new features and stay compatible with rules_haskell's ghcide
support.
* Fix Haddock warnings reported by ghcide
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Update rules_haskell
- Fixes the issue where a `sh_test` wrapping a `haskell_binary` couldn't
add runfiles as is the case with other Bazel rules.
CHANGELOG_BEGIN
CHANGELOG_END
* Save the runfiles environment at start-up
To work around the fact that `withProgName` overwrites `argv[0]`.
See https://gitlab.haskell.org/ghc/ghc/-/issues/18418.
* damlc_compile_test include damlc runfiles
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Buildifier now comes with a handy attachment to catch single `\`
characters inside strings and replace them with `\\` if the escape
sequence is invalid. Skylark/Python will do this at runtime anyway; this
just makes it clearer what the actual behavior is.
I needed to change `\` characters at the end of lines to `\\` manually
in order to stop Buildifier from simply concatenating the lines
together. Everything else was automatic.
CHANGELOG_BEGIN
CHANGELOG_END
* Update rules_haskell
* Pin stack_snapshot repositories
* Document stack_snapshot_json
CHANGELOG_BEGIN
CHANGELOG_END
* Don't pin stack_snapshot on Windows
The lock file is generated on Unix and includes unix specific
dependencies, e.g. `unix`. Most developers don't have easy access to a
Windows machine, so regenerating the lock file for Windows would be
inconvenient.
* upgrade stack 2.1.3 --> 2.3.1 on Windows
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Use Distroless for the Java Docker base image.
We switched away from Distroless because it was causing issues with
`docker pull` when you had Docker configured to use `gcloud` for
authentication, but weren't actually authenticated.
Adding `docker-credential-gcloud` to dev-env should hopefully fix this,
meaning we can switch back to a base image that is better-maintained.
CHANGELOG_BEGIN
CHANGELOG_END
* Bump rules_docker to v0.14.3.
This fixes an issue when running `bazel sync`:
```
ERROR: java.io.IOException: Error downloading [http://central.maven.org/maven2/javax/servlet/javax.servlet-api/3.0.1/javax.servlet-api-3.0.1.jar] to [...]/external/javax_servlet_api/javax.servlet-api-3.0.1.jar: Unknown host: central.maven.org
```
This is preparatory work for hitting the transaction service (and
others but that one is the most important) which isn’t possible via
DAML Script. To ease review, this PR only switches from a DAML Script
to a custom Scala binary. It does not yet change what is tested.
changelog_begin
changelog_end
* io.grpc:grpc-xxxx to 1.29.0 (from 1.22.1)
io.netty:netty-xxxx to .1.50.Final (from 4.1.37.Final)
io.nett.netty-tcp-native-boringssl-static to 2.0.30.Final (from 2.0.25.Final)
To resolve open vulnerabilities with these versions
netty-4.1.37.Final vulnerabilities
BDSA-2018-4022 (Medium)
BDSA-2019-2610 (Medium)
BDSA-2019-3119 (CVE-2019-16869) (Medium)
BDSA-2020-0130 (Medium)
BDSA BDSA-2019-4230 (CVE-2019-20445) (Low)
BDSA BDSA-2019-4231 (CVE-2019-20444) (Low)
BDSA BDSA-2020-0666 (CVE-2020-11612) (Low)
BDSA BDSA-2019-2642 (Low)
BDSA BDSA-2019-2649 (Low)
BDSA BDSA-2019-2643 (Low)
CHANGELOG_BEGIN
Upgrade io.grpc:grpc-xxxxx and io.netty:netty-xxx version to latest
released to avoid exposure to reported security vulnerabilities in
currently used versions
CHANGELOG_END
–
* Update spray versions to address vulnerabilities
CVE-2018-18853 and CVE-2018-18854
CHANGELOG_BEGIN
Upgrade io.grpc:grpc-xxxxx and io.netty:netty-xxx version to latest
released to avoid exposure to reported security vulnerabilities in
currently used versions
CHANGELOG_END
* do not change io.grpc version since reflection seems to be misbehaving
* Clarify how to bump grpc/netty/protobuf versions
Also "downgrade" netty to 4.1.48, according to
https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty
CHANGELOG_BEGIN
CHANGELOG_END
* Load protobuf deps after haskell deps to avoid loading an older version of rules_cc
* Upgrade protoc and protobuf-java to 3.11.0
* buildifier reformat
* regen unique int after rebase
* remove commented patch
Co-authored-by: Gerolf Seitz <gerolf.seitz@daml.com>
The workspace for the vendored node wrapper script `@nodejs_linux_amd64`
did previously not record a dependency on the nixpkgs provided node
workspace. This patch enforces that dependency by introducing a dummy
read of the vendored node binary.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* update rules_nixpkgs
* Use hermetic nixpkgs cc toolchain
CHANGELOG_BEGIN
CHANGELOG_END
* Work around Bazel's cc toolchain autodetection
* Use --crosstool_top for hermetic cc toolchain
When using --incompatible_enable_cc_toolchain_resolution instead
cc actions still depend on
`external/local_config_cc/builtin_include_directory_paths`
as well as
`external/nixpkgs_cc_toolchain_config/builtin_include_directory_paths`.
* override local_config_cc
* remove unused attribute
* Fix posix toolchain on Windows
* nixpkgs cc toolchain not on Windows
* Fix nixpkgs cc toolchain on MacOS
* nixpkgs cc toolchain uses bin/cc
* Use darwin.binutils on MacOS
* Remove clang(++) and gcc (g++) symlinks
The toolchain only considers `bin/cc` and having the other symlinks
around could lead to confusion
* Use hermetic toolchain in compatibility workspace
* Avoid empty linker flags
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
This is the first part of #5700
It adds tests that build create-daml-app using `daml build` and then
run the codegen and build the UI. Contrary to our main tests these
also run on Windows. This is actually reasonably simple by first
building the typescript libraries on Linux and then downloading them
on Windows.
There are two parts that are still missing from the tests in the main
workspace:
1. Building the extra feature. This should be fairly easy to add.
2. Running the pupeeter tests. At least MacOS and Linux should be
reasonably easy. I don’t know what horrors Windows will throw at
us. This step is what actually makes this a compatibility
test. Currently it doesn’t actually launch Sandbox and the JSON API.
Since this PR is already pretty large, I’d like to tackle those things
separately.
changelog_begin
changelog_end
This should fix the following issue that we see constantly on CI:
```
ld: warning: object file _ was built for newer OSX version (10.15) than being linked (10.14)
```
The issue was that the CC toolchain was not fully used in
haskell_cabal_package. --with-gcc (which is really --with-cc) only
applies when Cabal is calling the C compiler. However, in most cases
it is actually GHC itself which calls the C compiler. To make sure
that the right compiler is used in those cases, we have to pass
`-pgmc` and friends to GHC. This matches what rules_haskell does for
non cabal targets.
changelog_begin
changelog_end
* Update SDK versions in compatibility tests
This adds a Haskell script to generate a versions.bzl file that
contains the list of versions as well as their hashes. This should
make it a bit easier to keep things up2date going forward.
The script is a bit slow since downloading all the SDKs takes quite a
while but for now it should be good enough and is much more pleasant
than having to figure this out manually.
changelog_begin
changelog_end
* Address review comments
changelog_begin
changelog_end
* Fix excluded tests
changelog_begin
changelog_end
* docs: publish daml-cheat-sheet on docs.daml.com
* separate cheat sheet rule
* Fixed `@daml-cheat-sheet`
* Uses `_config.yml` to determine the source directory root.
* Uses `tar h` to resolve symbolic links. Otherwise the tarball just
contains symbolic links to the execroot.
* Uses flags to make `tar` and `gzip` reproducible, i.e. avoid
timestamps and the like.
* cleanup
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Fix ./fmt.sh check
We accidentally introduced hlint lints when we added the compatibility
workspace. The hlint code in ./fmt.sh correctly detects that there is
a hint but then exits with 0 since it uses the exit code from `echo`
rather than the one from `hlint`. The info that we printed with
`echo` didn’t actually make any sense so I added a new comment as to
why we run hlint here. This requires some setup changes so that the
compatibility workspace gets the same Haskell flags since otherwise
using the same hlint rules obviously does not make sense.
Fixes#5701
changelog_begin
changelog_end
* Update fmt.sh
Co-Authored-By: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Upgrade rules_nodejs to version 1.6.0
closes#5367
This includes the fixes for the issues in jest that we’ve been seeing.
changelog_begin
changelog_end
* Fix eslint rules
* A bit of progress
* Try to add LinkablePackageInfo (doesn’t seem to work yet)
* Add rootDirs
* revert da_ts_library
* da_ts_library: add LinkablePackageInfo info
* Remove react hook workaround
Since rules_nodejs 1.6.0 this fails with the following error:
```
● Test suite failed to run
Configuration error:
Could not locate module react mapped as:
/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js.
Please check your configuration for these entries:
{
"moduleNameMapper": {
"/^react$/": "/.../execroot/com_github_digital_asset_daml/bazel-out/k8-opt/bin/language-support/ts/daml-react/test.sh.runfiles/com_github_digital_asset_daml/node_modules/react/umd/react.development.js"
},
"resolver": null
}
49 | // like a promis without being one.
50 | /* eslint-disable @typescript-eslint/no-floating-promises */
> 51 | var react_1 = __importStar(require("react"));
| ^
52 | var react_hooks_1 = require("@testing-library/react-hooks");
53 | var index_1 = __importStar(require("./index"));
54 | var events_1 = require("events");
at createNoMappedModuleFoundError (../../../../../../../../../../../node_modules/jest-resolve/build/index.js:501:17)
at Object.<anonymous> (index.test.js:51:28)
Test Suites: 1 failed, 1 total
Tests: 0 total
Snapshots: 0 total
Time: 1.88s
Ran all test suites within paths "language-support/ts/daml-react/DamlLedger.d.ts", "language-support/ts/daml-react/DamlLedger.js", "language-support/ts/daml-react/context.d.ts", "language-support/ts/daml-react/context.js", "language-support/ts/daml-react/hooks.d.ts", "language-support/ts/daml-react/hooks.js", "language-support/ts/daml-react/index.d.ts", "language-support/ts/daml-react/index.js", "language-support/ts/daml-react/index.test.d.ts", "language-support/ts/daml-react/index.test.js".
=
```
* rootDirs is not needed for tsc
This is only required for ts_project
* Update yarn Bazel packages
* docs/theme add missing dependencies
* Remove unused attribute module_root
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
This used to be a dependency of our zip library that you couldn’t
disable. However, in the meantime it has gotten a flag to disable
it and we don’t actually use it. Given that the upstream URL is dead
atm now seems like a good time to kill it.
changelog_begin
changelog_end
* Update rules_haskell
The workaround for linking against `Cffi` in the REPL has been
upstreamed in a more generalized form.
CHANGELOG_BEGIN
CHANGELOG_END
* ghcide: Use rules_haskell's hie-bios support
* Document `ghcide` Bazel integration
* Rename files to match module names
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Includes a change that should reduce the size of runfiles manifest files
and runfiles trees for targets with runtime dependencies on the Python
toolchain.
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
* Bump rules_haskell
Still checking if that helps with GHC 8.8 but we should upgrade this
either way.
changelog_begin
changelog_end
* disable grpc patch
* shut up buildifier
* delete unused ghci grpc patch
* Fix Cffi library not found issues
* Update deps.bzl
Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* Disable http2 with Nix to work around segfaults
CHANGELOG_BEGIN
CHANGELOG_END
* Disable http2 in dev-env calls to nix-build as well
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
* Update rules_haskell
* Includes Bazel 2.0.0 support
* Some patches have been upstreamed
CHANGELOG_BEGIN
CHANGELOG_END
* External haskell_cabal_binary|library verbose = False
`haskell_cabal_binary|library` now supports setting `verbose = False`
which also avoids warnings coming from `Setup.hs`. As these are external
dependencies we are not going to address these warnings anyway. So, they
are just noise. This makes it unnecessary to pass `-w` or `-optF=-w`.
In case of build failure all errors and warnings will be displayed.
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
The `sandbox-perf` build has been failing for a while with the following
errors:
```
INFO: From Generating benchmark code for //ledger/sandbox-perf:sandbox-perf_codegen:
JMH benchmark generation: JMH Benchmark generator failed
JMH benchmark generation: Benchmark classes should not be final.
[com.digitalasset.platform.sandbox.perf.LargeTransactionBench]
JMH benchmark generation: The instantiated @State class cannot be abstract.
[com.digitalasset.platform.sandbox.perf.PerfBenchState]
```
However, these errors are ignored; running the benchmarks runs the
`AcsBench` benchmark and ignores the fact `LargeTransactionBench` and
`SimpleBench` failed to compile.
I've fixed the errors by making sure that classes are not final, and
that `SimpleBench` uses a concrete state class.
This also introduces a cheap patch to the Scala JMH Bazel rules that
makes sure they fail if there are any errors.
I'm not really sure how to patch the Bazel rules properly, but someone
else might have an idea. :-)
CHANGELOG_BEGIN
CHANGELOG_END
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Remove unused npm_nix
* Use node_nix directly on Unix
rules_nodejs requires a vendored node to contain the node distribution
in a subdirectory of the provided external repository. With the default
node Nix package this is not the case. The nodejs_dev_env workspace used
an intermediate repository rule to fulfill these requirements.
Occasionally, this caused flakiness on CI.
This change fulfills this requirement on the Nix side, so that Bazel can
consume the provided Nix derivation directly.
CHANGELOG_BEGIN
CHANGELOG_END
* Avoid collision with dev-env tools like license-checker
* buildifier-fix
* Add comment motivating use of `@node_nix` on Unix.
* Document npm-cli patch
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
* language: bazel rules for daml-json-types/daml-ledger-fetch
This moves the daml-json-types/daml-ledger-fetch libraries out of the
tests directory and builds them with bazel. We'll rename these libraries
in a follow up PR.
CHANGELOG_BEGIN
CHANGELOG_END
* Update deps.bzl
Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* updated package.json
* rename nodejs patch
* update yarn.lock
* update @bazel/bazel dependency
* wrong typescript version in toplevel package.json
Co-authored-by: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
This PR fixes a minor issue where we were including more references
than we need which causes problems on older SDKs where GHC.Prim is not
a stable module since GHC.Prim cannot be imported (it’s builtin GHC
magic).
More importantly, this PR adds a test for cross-SDK data-dependencies
against the DAVL DAR which is built with SDK 0.13.32 before we started
making all the changes. The test also includes a slightly hacky but
mostly reasonable way of actually using the templates instead of just
the data types from the DAVL DAR which and a scenario to test this.
changelog_begin
changelog_end
