This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.
As announced, this will be merged on Saturday to avoid too many conflicts.
changelog_begin
changelog_end
* Docs restyle WIP
* Minor color changes
* Adjust fonts and shadows
* Remove redundant local contents
* Further fixes to "In this Section"
* Restyle previous and next buttons
* Make visited link color blue
* Top-level nav
* Change how anchors are underlined
* Fix header spacing
* Improve tables
* Improve search results
* Improve copy button
Match sphinx versions between dev-env and bazel
* Copy button and code block fixes
* Fix copy button z-index
* Making things reactive
* Make the title spacing work on mobile & tablet
* Pixel adjustment for S3 hosting
* Spacing and font-size fixes
* Fix content menu for tablet
* Tidy up menu indentation
* Reorder Mobile Menu
* Move unified nav up
CHANGELOG_BEGIN
[Docs] New styling
CHANGELOG_END
* Remove non-free fonts
* Fix button padding
* Fix spacing above first h2
* Fix pixel adjustment of h1 line
* Fix menu click effects and antialiasing
* Pixel adjust h1 line once more
We use `xargs` a lot but rely on the version on the user's system.
This adds it to dev-env so it's consistent everywhere.
If nothing else, this means we can safely use the `--no-run-if-empty`
flag, which is not available on BSD (and therefore macOS) `xargs`.
CHANGELOG_BEGIN
CHANGELOG_END
The version in nixpkgs is v2.12.10, which means that the Scala REPL we
used was not the same version as we use in our code.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-on-sql: Formatting fixes for the README.
* daml-on-sql: Rename to "DAML on PostgreSQL" within the README.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-on-sql: Format README to make Markdown conversion nicer.
Markdown conversion:
pandoc --reference-links --reference-location=section -t markdown+backtick_code_blocks-fenced_code_attributes -o ledger/daml-on-sql/README.md ledger/daml-on-sql/README.rst && sed -i 's/\\//g' ledger/daml-on-sql/README.md
* daml-on-sql: It's "DAML for PostgreSQL", not "on PostgreSQL" now.
* daml-on-sql: Generate a Markdown README with a script.
This is easier to parse by human beings than the reStructuredText.
* daml-on-sql: Reflow the README and manipulate whitespace a little.
* Upgrade nixpkgs revision
* Remove unused minio
It used to be used as a gateway to push the Nix cache to GCS, but has
since been replaced by nix-store-gcs-proxy.
* Update Bazel on Windows
changelog_begin
changelog_end
* Fix hlint warnings
The nixpkgs update implied an hlint update which enabled new warnings.
* Fix "Error applying patch"
Since Bazel 2.2.0 the order of generating `WORKSPACE` and `BUILD` files
and applying patches has been reversed. The allows users to define
patches to these files that will not be immediately overwritten.
However, it also means that patches on another repository's original
`WORKSPACE` file will likely become invalid.
* a948eb7255
* https://github.com/bazelbuild/bazel/issues/10681
Hint: If you're generating a patch with `git` then you can use the
following command to exclude the `WORKSPACE` file.
```
git diff ':(exclude)WORKSPACE'
```
* Update rules_nixpkgs
* nixpkgs location expansion escaping
* Drop --noincompatible_windows_native_test_wrapper
* client_server_test using sh_inline_test
client_server_test used to produce an executable shell script in form of
a text file output. However, since the removal of
`--noincompatible_windows_native_test_wrapper` this no longer works on
Windows since `.sh` files are not directly executable on Windows.
This change fixes the issue by producing the script file in a dedicated
rule and then wrapping it in a `sh_test` rule which also works on
Windows.
* daml_test using sh_inline_test
* daml_doc_test using sh_inline_test
* _daml_validate_test using sh_inline_test
* damlc_compile_test using sh_inline_test
* client_server_test find .exe on Windows
* Bump Windows cache for Bazel update
Remove `clean --expunge` after merge.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Use Distroless for the Java Docker base image.
We switched away from Distroless because it was causing issues with
`docker pull` when you had Docker configured to use `gcloud` for
authentication, but weren't actually authenticated.
Adding `docker-credential-gcloud` to dev-env should hopefully fix this,
meaning we can switch back to a base image that is better-maintained.
CHANGELOG_BEGIN
CHANGELOG_END
* Bump rules_docker to v0.14.3.
This fixes an issue when running `bazel sync`:
```
ERROR: java.io.IOException: Error downloading [http://central.maven.org/maven2/javax/servlet/javax.servlet-api/3.0.1/javax.servlet-api-3.0.1.jar] to [...]/external/javax_servlet_api/javax.servlet-api-3.0.1.jar: Unknown host: central.maven.org
```
* Copy code button
Added the sphinx copy code extension (https://sphinx-copybutton.readthedocs.io/en/latest/) to the docs. Tested with and it worked.
CHANGELOG_START
CHANGELOG_END
* Remove vendored copy of sphinx_copybutton
changelog_begin
changelog_end
* remove debugging output
changelog_begin
changelog_end
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
It doesn't seem to be used anywhere. Obviously `git grep jo` returns a
lot of results, so I may have missed something in the noise, but I did a
reasonable effort to look through them.
CHANGELOG_BEGIN
CHANGELOG_END
This PR upgrades the dependencies of sphinx-autobuild to address
security vulnerabilities. I tested on Linux that the docs preview
script which is the only user of sphinx-autobuild still works
correctly.
I’ve renamed default.nix to requirements.nix to match the name used by
pypi2nix (otherwise you just have to rename it ant the end which is a
bit stupid). I’ve also added a brief readme.
I would appreciate if somebody could test the docs preview script on
macos.
fixes#6317
changelog_begin
changelog_end
The issues underlying #6173 prompted me to look into which `find`
version we had in dev-env, and I was surprised to notice we had none. We
already have `findutils` in our nix configuration, however, so this is
just adding the symlink.
Note that, on my machine at least, switching from the macOS-provided one
to the dev-env one does not change the result order for the hash
calculation in `ci/patch_bazel_windows`, so this would likely not have
helped for #6173. Still, we use `find` in many places in our scripts so
I think it's worth having there.
CHANGELOG_BEGIN
CHANGELOG_END
* Upgrade puppeteer
We’ve seen a couple of issues in the compatibility tests of the form
```
Error: Protocol error (Runtime.callFunctionOn): Target closed.
```
Looking at the issue tracker in puppeteer this might be fixed in newer
versions and I don’t see why we should stick to a fairly old version
anyway.
changelog_begin
changelog_end
* Upgrade nodejs
changelog_begin
changelog_end
* temporary add a step to kill node_modules
changelog_begin
changelog_end
* Kill live server and try to fix Windows
changelog_begin
changelog_end
* Undo rm
changelog_begin
changelog_end
* io.grpc:grpc-xxxx to 1.29.0 (from 1.22.1)
io.netty:netty-xxxx to .1.50.Final (from 4.1.37.Final)
io.nett.netty-tcp-native-boringssl-static to 2.0.30.Final (from 2.0.25.Final)
To resolve open vulnerabilities with these versions
netty-4.1.37.Final vulnerabilities
BDSA-2018-4022 (Medium)
BDSA-2019-2610 (Medium)
BDSA-2019-3119 (CVE-2019-16869) (Medium)
BDSA-2020-0130 (Medium)
BDSA BDSA-2019-4230 (CVE-2019-20445) (Low)
BDSA BDSA-2019-4231 (CVE-2019-20444) (Low)
BDSA BDSA-2020-0666 (CVE-2020-11612) (Low)
BDSA BDSA-2019-2642 (Low)
BDSA BDSA-2019-2649 (Low)
BDSA BDSA-2019-2643 (Low)
CHANGELOG_BEGIN
Upgrade io.grpc:grpc-xxxxx and io.netty:netty-xxx version to latest
released to avoid exposure to reported security vulnerabilities in
currently used versions
CHANGELOG_END
–
* Update spray versions to address vulnerabilities
CVE-2018-18853 and CVE-2018-18854
CHANGELOG_BEGIN
Upgrade io.grpc:grpc-xxxxx and io.netty:netty-xxx version to latest
released to avoid exposure to reported security vulnerabilities in
currently used versions
CHANGELOG_END
* do not change io.grpc version since reflection seems to be misbehaving
* Clarify how to bump grpc/netty/protobuf versions
Also "downgrade" netty to 4.1.48, according to
https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty
CHANGELOG_BEGIN
CHANGELOG_END
* Load protobuf deps after haskell deps to avoid loading an older version of rules_cc
* Upgrade protoc and protobuf-java to 3.11.0
* buildifier reformat
* regen unique int after rebase
* remove commented patch
Co-authored-by: Gerolf Seitz <gerolf.seitz@daml.com>
It should not be an issue, but I know of at least 4 people for whom
having envsubst in dev-env causes their local git to get confused and
spit out many lines of warning on each invocation. This also seems to
make git much slower.
Given that we're only using envsubst in this one place, I think it may
be worth replacing.
CHANGELOG_BEGIN
CHANGELOG_END
First version of static verification tool.
The current state of the tool:
- Reads DAR files.
- Partially evaluates the code.
- Generates constraints for the field and choice to be verified.
- Passes the constraints to an SMT solver.
- Some basic tests.
* update rules_nixpkgs
* Use hermetic nixpkgs cc toolchain
CHANGELOG_BEGIN
CHANGELOG_END
* Work around Bazel's cc toolchain autodetection
* Use --crosstool_top for hermetic cc toolchain
When using --incompatible_enable_cc_toolchain_resolution instead
cc actions still depend on
`external/local_config_cc/builtin_include_directory_paths`
as well as
`external/nixpkgs_cc_toolchain_config/builtin_include_directory_paths`.
* override local_config_cc
* remove unused attribute
* Fix posix toolchain on Windows
* nixpkgs cc toolchain not on Windows
* Fix nixpkgs cc toolchain on MacOS
* nixpkgs cc toolchain uses bin/cc
* Use darwin.binutils on MacOS
* Remove clang(++) and gcc (g++) symlinks
The toolchain only considers `bin/cc` and having the other symlinks
around could lead to confusion
* Use hermetic toolchain in compatibility workspace
* Avoid empty linker flags
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
This PR adds a simple daily job that runs the performance test on a
chosen "baseline" commit and then runs the same benchmark on latest
master. This should allow us to track overall performance improvements.
CHANGELOG_BEGIN
CHANGELOG_END
* docs: publish daml-cheat-sheet on docs.daml.com
* separate cheat sheet rule
* Fixed `@daml-cheat-sheet`
* Uses `_config.yml` to determine the source directory root.
* Uses `tar h` to resolve symbolic links. Otherwise the tarball just
contains symbolic links to the execroot.
* Uses flags to make `tar` and `gzip` reproducible, i.e. avoid
timestamps and the like.
* cleanup
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Diff with messaging feature and some noise manually removed
* Bazel target to use patch file in other build targets
* Patch file as data dep for integration tests
* Attempt to patch and test messaging feature in create-daml-app test
changelog_begin
changelog_end
* Use exports_files instead of filegroup
* Remove file existence checks that don't make sense
* Add patch to dev_env and reference it from integration tests
* Include patch on windows for later
* Set up yarn env again after codegen
* Restore file check
* Fix typo in comment on util function
* Add Tasty steps to make process explicit
* Use messaging patch for code snippets in GSG
* Use messaging code from template instead of copy
* Remove copied message code
* Refactor script to copy template code with messaging patch
* No need to retry second yarn install (only local deps should be updated)
changelog_begin
changelog_end
For now this is only used for the daml-helper tests. I’ll shuffle
things around and use it for all tests in a separate PR.
* Include Bazel patch to mark tests as exclusive
This should hopefully avoid rerunning the conformance tests as often
as we do now. While this patch is not applied on Windows (since we get
it from nix), this is not really an issue since most of the exclusive
tests (in particular, all conformance tests) are disabled on Windows
anyway.
I’ve tested this locally accross a couple of runs and I get the
caching I want and looking at the code in the patch, the change looks
very reasonable. I somewhat wonder if it just broke internally at
google because they marked tests as exclusive that should have gotten
no-cache.
changelog_begin
changelog_end
* Disable caching for canton
This reduces the number of GHCs to 2 on Linux (regular and DWARF) and
1 on macOS. Given that each derivation is > 1 GB this should hopefully
help a bit.
changelog_begin
changelog_end
This disables the PDF docs builds on MacOS on CI (they are still built
locally by default) and removes them from the Nix closure by
introducing a separate ci-cached attribute that filters out texlive.
Since we built `nix-build nix -A tools -A cached` on CI, I’ve also
removed all the Tex stuff from tools which only means that it ends up
in PATH which nobody seems to care about.
changelog_begin
changelog_end
* Remove unused npm_nix
* Use node_nix directly on Unix
rules_nodejs requires a vendored node to contain the node distribution
in a subdirectory of the provided external repository. With the default
node Nix package this is not the case. The nodejs_dev_env workspace used
an intermediate repository rule to fulfill these requirements.
Occasionally, this caused flakiness on CI.
This change fulfills this requirement on the Nix side, so that Bazel can
consume the provided Nix derivation directly.
CHANGELOG_BEGIN
CHANGELOG_END
* Avoid collision with dev-env tools like license-checker
* buildifier-fix
* Add comment motivating use of `@node_nix` on Unix.
* Document npm-cli patch
Co-authored-by: Andreas Herrmann <andreash87@gmx.ch>
* Get grpc from nix on unix
The one from Bazel seems to cause linking issues when trying to run
things in GHCi. I’ve spent some time trying to use rules_foreign_cc to
build gRPC using CMake but decided that for my own sanity it’s better
to not pursue that further.
* Address review comments
* Add missing module load
* Cleanup GHCI_SCRIPT
* use the correct file ending on macos
* Import is_linux
* Switch back to grpc-1.23
The newer version seems to cause issues in combination with the java libraries.
* Try to fix package_app on macos
* more debugging
* Maybe this is not necessary, we will never know
* linkers are the worst
* Remove debugging output again
* readd rpaths
* treat libdispatch specially
* remove hack
* more fooling around
* lalala
* Fix pip and pipenv
Since the latest nixpkgs update pip is no longer contained in the python
derivation but instead in its own `python37Packages.pip` derivation.
Additionally the pipenv version in nixpkgs is incompatible with the pip
version, as pipenv expects a module to be present which is only
available in later versions. This change overrides pip with a version
update to fix this issue.
Upstream nixpkgs has already updated pip accordingly
* Update nixpkgs
The versions of `pipenv` and `pip` contained in `nixpkgs` where
previously incompatible. They are compatible in the new `nixpkgs`
revision.
Just overriding the pip version caused too many uncached rebuilds and
timed out on CI.
* Update nixpkgs further
To incorporate
efce3c1367
* Try to build fewer GHCs
* nix: drop sass override
nixpkgs-unstable is on the same page.
* nix: use upstream chromedriver
* nix: remove pex override
* nix: drop pypi2nix override
* Fix bazel-watcher
* try upgrading rules go
* Remove overrides for ibazel
* Upgrade rules_go
* Remove references to sass overrides
* ledger-api-test-tool-on-canton: Run Canton.
* ledger-api-test-tool-on-canton: Run the SemanticTests against Canton.
This was _so_ much work.
These will probably be flaky, so can't merge them in until we fix the
underlying issues with the tests around multi-participant allocation.
* ledger-api-test-tool: Wait for parties to arrive on all participants.
Thanks to Canton for exacerbating this bug.
Can't turn on all the tests on Canton yet as there are other spurious
failures. Will investigate next.
* ledger-api-test-tool: Move all contract key tests to ContractKeys.
If a ledger doesn't support ContractKeys, they need to be able to turn
these tests off.
* ledger-api-test-tool: If a test name is misspelled, fail immediately.
I keep getting names slightly wrong in the `--exclude` arguments and
wondering why it didn't work.
* client_server: Revert client_server_test.bzl.
Turns out I don't know what I'm doing. ¯\_(ツ)_/¯
* sandbox: Don't call `.toString` on an array. Doesn't do much.
* dev-env: Don't untar netcat automatically.
It needs to be installed with Pacman.
* Use cc_wrapper in ghcide test
The cc_wrapper is needed to find library dependencies.
* Update rules_haskell
* update rules_nixpkgs
Fixes issue with `nixpkgs_local_repository` and Nix `import`.
* Shorten RULES_HASKELL_EXEC_ROOT
Cabal package library paths are longer and overflow the command-line
length when calling `cc` on MacOS for template Haskell dependencies in
the IDE tests. Shortening the `RULES_HASKELL_EXEC_ROOT` prefix for each
`(LD_)LIBRARY_PATH` entry works around the issue.
* bazel: 0.28.1 --> 1.1.0
* bazel-watcher sha256
* Fix missing line in patch
* proto_source_root --> strip_import_prefix
See https://github.com/bazelbuild/bazel/issues/7153 for details.
* Update rules_nixpkgs
Required to avoid errors of the form
```
ERROR: An error occurred during the fetch of repository 'node_nix':
parameter 'sep' may not be specified by name, for call to method split(sep, maxsplit = None) of 'string'
```
and
```
ERROR: An error occurred during the fetch of repository 'node_nix':
Traceback (most recent call last):
File "/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 149
_execute_or_fail(repository_ctx, <3 more arguments>)
File "/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 318, in _execute_or_fail
fail(<1 more arguments>)
Cannot build Nix attribute 'nodejs'.
Command: [/Users/runner/.nix-profile/bin/nix-build, /private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/node_nix/nix/bazel.nix, "-A", "nodejs", "--out-link", "bazel-support/nix-out-link", "-I", "nixpkgs=/private/var/tmp/_bazel_runner/17d2b3954f1c6dcf5414d5453467df9a/external/nixpkgs/nixpkgs"]
Return code: 1
Error output:
src/main/tools/process-tools.cc:173: "setitimer": Invalid argument
```
* Update rules_scala
* .proto has been removed, use [ProtoInfo] instead
See
https://docs.bazel.build/versions/1.1.0/be/protocol-buffer.html#proto_library
* python3_nix add nix_file attribute
To avoid the following error
```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/BUILD:66:1: //:nix_python3_runtime depends on @python3_nix//:bin/python in repository @python3_nix which failed to fetch. no such package '@python3_nix//': Traceback (most recent call last):
File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 149
_execute_or_fail(repository_ctx, <3 more arguments>)
File "/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/io_tweag_rules_nixpkgs/nixpkgs/nixpkgs.bzl", line 318, in _execute_or_fail
fail(<1 more arguments>)
Cannot build Nix attribute 'python3'.
Command: [/home/aj/.nix-profile/bin/nix-build, "-E", "import <nixpkgs> { config = {}; overlays = []; }", "-A", "python3", "--out-link", "bazel-support/nix-out-link", "-I", "nixpkgs=/home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/nixpkgs/nixpkgs"]
Return code: 1
Error output:
error: anonymous function at /home/aj/.cache/bazel/_bazel_aj/5f825ad28f8e070f999ba37395e46ee5/external/nixpkgs/nixpkgs.nix:3:1 called with unexpected argument 'config', at (string):1:1
```
* rules_haskell unnamed string.split(_, maxsplit = _)
The keyword argument may no longer be named.
* string.replace(_, _, maxsplit = _) may not be named
* Move proto sources from deps to data
Fixes
```
ERROR: /home/aj/tweag.io/da/da-bazel-1.1/daml-lf/archive/BUILD.bazel:150:1: in deps attribute of scala_test rule //daml-lf/archive:daml_lf_archive_reader_tests_test_suite_src_test_scala_com_digitalasset_daml_lf_archive_DecodeV1Spec.scala: '//daml-lf/archive:daml_lf_1.6_archive_proto_srcs' does not have mandatory providers: 'JavaInfo'. Since this rule was created by the macro 'da_scala_test_suite', the error might have been caused by the macro implementation
```
* Define sha256 for haskell_ghc__paths
Bazel 1.1.0 fails on missing hashes.
* Disable --incompatible_windows_native_test_wrapper
* //compiler/daml-extension don't modify sources
Modifying sources in-place can cause issues on Windows, where build
actions are not sandboxed and changes on sources can affect other build
steps.
* bazel-genfiles --> bazel-bin
The bazel-genfiles symlink has been removed since Bazel 1.0.
See https://github.com/bazelbuild/bazel/issues/8651
* Mark dev_env_tool repository rule as configure
See
https://docs.bazel.build/versions/1.1.0/skylark/lib/globals.html#repository_rule
* Move data deps into data attribute
* Mark dev_env_tool as local = True
* Manually fetch @makensis_dev_env
