* Test case for LockedFreePort not colliding with port 0
changelog_begin
changelog_end
* Discover dynamic port range on Linux
* Random port generator outside ephemeral range
* remove dev comments
* Draw FreePort from outside the ephemeral port range
Note, there is a race condition between the socket being closed and the
lock-file being created in LockedFreePort. This is not a new issue, it
was already present with the previous port 0 based implementation.
LockedFreePort handles this by attempting to find a free port and taking
a file lock multiple times.
But, it could happen that A `find`s port N, and obtains the lock, but
doesn't bind port N again, yet; then B binds port N during `find`; then
A attempts to bind port N before B could release it again and fails
because B still holds it.
* Select dynamic port range based on OS
* Detect dynamic port range on MacOS and Windows
* Import sysctl from Nix on MacOS
changelog_begin
changelog_end
* Windows line separator
* FreePort helpers visibility
* Use more informative exception types
* Use a more light weight unit test
* Add comments
* Fix Windows
* Update libs-scala/ports/src/main/scala/com/digitalasset/ports/FreePort.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Update libs-scala/ports/src/main/scala/com/digitalasset/ports/FreePort.scala
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* Add a comment to clarify the generated port range
* fmt
* unused import
* Split libs-scala/ports
Splits the FreePort and LockedFreePort components into a separate
library as this is only used for testing purposes.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
Co-authored-by: Stefano Baghino <43749967+stefanobaghino-da@users.noreply.github.com>
* vanilla job test on main pipeline
changelog_begin
changelog_end
* move job to daily compat tests
* add timeout to dev-env and changes based on code review
* Update the channels link in Upgrading.md
* Update the nixpkgs-unstable commit
CHANGELOG_BEGIN
CHANGELOG_END
* Fix new `hlint` warnings.
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
CHANGELOG_BEGIN
* [Integration Kit] Removed trace_context field from Ledger API and its bindings as we now have trace context propagation support via gRPC metadata. If you are constructing or consuming Ledger API requests or responses directly, you may need to update your code.
CHANGELOG_END
This is still a bit rough. It currently retries regardless of the
error (however, with an exponential backoff and a limit so not that
big of an issue, at worst we waste some time retrying). We also print
the exception to stderr which is a bit nasty but useful for debugging.
I did verify in logs that we hit the retry code path and that things
seem to be properly cached.
I’ll clean this up & upstream this to Bazel once we have verified for
a few days/weeks that it works well for us.
Reviewing patch files is nasty so I recommend to take a look at the
branch in my Bazel fork instead
https://github.com/cocreature/bazel/tree/cache-retry.
changelog_begin
changelog_end
* Introduce shell.nix to expose the Nix tooling, if required.
* dev-env: Rewrite the ghcide script to use nix-shell.
* language-support/hs: Use `nix-shell` for export-package.sh.
* Nix: Add a shebang to the Bazel wrapper script.
CHANGELOG_BEGIN
CHANGELOG_END
* dev-env: Standardize the `set` header in `ghcide`.
* dev-env: Replace `gpg` and `make` with symlinks.
* dev-env: Remove sphinx-autobuild. It's brought in by pipenv.
* dev-env: Remove Python 3.6 references. They seem to be unused.
CHANGELOG_BEGIN
CHANGELOG_END
* Upgrade Scala 2.12 to v2.12.13.
This is being pulled in anyway because of Maven/Gradle/etc's fun
"favor the most recent" resolution mechanism. The version of Akka we
depend upon transitively depends on Scala 2.12.13, so any downstream
consumers will see that as the Scala version required.
Bringing the Daml repo in line means no more confusion.
CHANGELOG_BEGIN
- [Scala Bindings] If you are using Daml on Scala 2.12, it now depends
on Scala v2.12.13 (from v2.12.12).
CHANGELOG_END
* Scala 2.12.13 is the default version in our pinned version of nixpkgs.
* Upgrade Scala 2.13's Wartremover version.
* Rename `scala_version_rule` to `scala_version_configure`.
* Add a test case to ensure the Scala versions are the same everywhere.
* Add tests for the Scala JAR versions in maven_install_*.json
* gatling-utils: Change the sort order of the expected CSV in tests.
I don't know why this changed, but it seems to be stable.
* compatibility: `scala_version` -> `scala_version_configure`.
* Bazel: Disable the Scala version tests on Windows.
* compatibility: Upgrade Wartremover to Scala 2.12.13.
This fixes Scaladoc and our pom file generation.
It also clears up the confusing error around gatling and removes a
redundant dependency on sbt (no idea why we had that in the first
place) both of which resulted in Scala 2.12 dependencies in our 2.13
lockfile which is obviously bad.
With this, we should now be ready to publish Scala 2.13 artifacts once
the ledger API test tool PR lands.
changelog_begin
changelog_end
It relies on a Bintray URL, which is likely to break soon as Bintray
sunsets.
It's also not been referenced by anything in this repo since at least
open-sourcing, so the value it provides seems low. There's no
documentation of why it's here and what its planned usage was, but based
on the [project page](https://github.com/jeremylong/DependencyCheck) I
imagine it's been supplanted by BlackDuck now.
CHANGELOG_BEGIN
CHANGELOG_END
Details are in the comments but this seems to workaround the annoying
tls issues we have been dealing with for the past few days.
changelog_begin
changelog_end
Add the https://github.com/google/google-java-format code formatting tool to dev-env.
Uses the name javafmt to call it, to mirror scalafmt.
Making it part of our checks will be done in a follow-up PR to make this easy to review.
changelog_begin
changelog_end
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.
As announced, this will be merged on Saturday to avoid too many conflicts.
changelog_begin
changelog_end
* Docs restyle WIP
* Minor color changes
* Adjust fonts and shadows
* Remove redundant local contents
* Further fixes to "In this Section"
* Restyle previous and next buttons
* Make visited link color blue
* Top-level nav
* Change how anchors are underlined
* Fix header spacing
* Improve tables
* Improve search results
* Improve copy button
Match sphinx versions between dev-env and bazel
* Copy button and code block fixes
* Fix copy button z-index
* Making things reactive
* Make the title spacing work on mobile & tablet
* Pixel adjustment for S3 hosting
* Spacing and font-size fixes
* Fix content menu for tablet
* Tidy up menu indentation
* Reorder Mobile Menu
* Move unified nav up
CHANGELOG_BEGIN
[Docs] New styling
CHANGELOG_END
* Remove non-free fonts
* Fix button padding
* Fix spacing above first h2
* Fix pixel adjustment of h1 line
* Fix menu click effects and antialiasing
* Pixel adjust h1 line once more
We use `xargs` a lot but rely on the version on the user's system.
This adds it to dev-env so it's consistent everywhere.
If nothing else, this means we can safely use the `--no-run-if-empty`
flag, which is not available on BSD (and therefore macOS) `xargs`.
CHANGELOG_BEGIN
CHANGELOG_END
The version in nixpkgs is v2.12.10, which means that the Scala REPL we
used was not the same version as we use in our code.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-on-sql: Formatting fixes for the README.
* daml-on-sql: Rename to "DAML on PostgreSQL" within the README.
CHANGELOG_BEGIN
CHANGELOG_END
* daml-on-sql: Format README to make Markdown conversion nicer.
Markdown conversion:
pandoc --reference-links --reference-location=section -t markdown+backtick_code_blocks-fenced_code_attributes -o ledger/daml-on-sql/README.md ledger/daml-on-sql/README.rst && sed -i 's/\\//g' ledger/daml-on-sql/README.md
* daml-on-sql: It's "DAML for PostgreSQL", not "on PostgreSQL" now.
* daml-on-sql: Generate a Markdown README with a script.
This is easier to parse by human beings than the reStructuredText.
* daml-on-sql: Reflow the README and manipulate whitespace a little.
* Upgrade nixpkgs revision
* Remove unused minio
It used to be used as a gateway to push the Nix cache to GCS, but has
since been replaced by nix-store-gcs-proxy.
* Update Bazel on Windows
changelog_begin
changelog_end
* Fix hlint warnings
The nixpkgs update implied an hlint update which enabled new warnings.
* Fix "Error applying patch"
Since Bazel 2.2.0 the order of generating `WORKSPACE` and `BUILD` files
and applying patches has been reversed. The allows users to define
patches to these files that will not be immediately overwritten.
However, it also means that patches on another repository's original
`WORKSPACE` file will likely become invalid.
* a948eb7255
* https://github.com/bazelbuild/bazel/issues/10681
Hint: If you're generating a patch with `git` then you can use the
following command to exclude the `WORKSPACE` file.
```
git diff ':(exclude)WORKSPACE'
```
* Update rules_nixpkgs
* nixpkgs location expansion escaping
* Drop --noincompatible_windows_native_test_wrapper
* client_server_test using sh_inline_test
client_server_test used to produce an executable shell script in form of
a text file output. However, since the removal of
`--noincompatible_windows_native_test_wrapper` this no longer works on
Windows since `.sh` files are not directly executable on Windows.
This change fixes the issue by producing the script file in a dedicated
rule and then wrapping it in a `sh_test` rule which also works on
Windows.
* daml_test using sh_inline_test
* daml_doc_test using sh_inline_test
* _daml_validate_test using sh_inline_test
* damlc_compile_test using sh_inline_test
* client_server_test find .exe on Windows
* Bump Windows cache for Bazel update
Remove `clean --expunge` after merge.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* Use Distroless for the Java Docker base image.
We switched away from Distroless because it was causing issues with
`docker pull` when you had Docker configured to use `gcloud` for
authentication, but weren't actually authenticated.
Adding `docker-credential-gcloud` to dev-env should hopefully fix this,
meaning we can switch back to a base image that is better-maintained.
CHANGELOG_BEGIN
CHANGELOG_END
* Bump rules_docker to v0.14.3.
This fixes an issue when running `bazel sync`:
```
ERROR: java.io.IOException: Error downloading [http://central.maven.org/maven2/javax/servlet/javax.servlet-api/3.0.1/javax.servlet-api-3.0.1.jar] to [...]/external/javax_servlet_api/javax.servlet-api-3.0.1.jar: Unknown host: central.maven.org
```
* Copy code button
Added the sphinx copy code extension (https://sphinx-copybutton.readthedocs.io/en/latest/) to the docs. Tested with and it worked.
CHANGELOG_START
CHANGELOG_END
* Remove vendored copy of sphinx_copybutton
changelog_begin
changelog_end
* remove debugging output
changelog_begin
changelog_end
Co-authored-by: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
It doesn't seem to be used anywhere. Obviously `git grep jo` returns a
lot of results, so I may have missed something in the noise, but I did a
reasonable effort to look through them.
CHANGELOG_BEGIN
CHANGELOG_END
This PR upgrades the dependencies of sphinx-autobuild to address
security vulnerabilities. I tested on Linux that the docs preview
script which is the only user of sphinx-autobuild still works
correctly.
I’ve renamed default.nix to requirements.nix to match the name used by
pypi2nix (otherwise you just have to rename it ant the end which is a
bit stupid). I’ve also added a brief readme.
I would appreciate if somebody could test the docs preview script on
macos.
fixes#6317
changelog_begin
changelog_end
The issues underlying #6173 prompted me to look into which `find`
version we had in dev-env, and I was surprised to notice we had none. We
already have `findutils` in our nix configuration, however, so this is
just adding the symlink.
Note that, on my machine at least, switching from the macOS-provided one
to the dev-env one does not change the result order for the hash
calculation in `ci/patch_bazel_windows`, so this would likely not have
helped for #6173. Still, we use `find` in many places in our scripts so
I think it's worth having there.
CHANGELOG_BEGIN
CHANGELOG_END
* Upgrade puppeteer
We’ve seen a couple of issues in the compatibility tests of the form
```
Error: Protocol error (Runtime.callFunctionOn): Target closed.
```
Looking at the issue tracker in puppeteer this might be fixed in newer
versions and I don’t see why we should stick to a fairly old version
anyway.
changelog_begin
changelog_end
* Upgrade nodejs
changelog_begin
changelog_end
* temporary add a step to kill node_modules
changelog_begin
changelog_end
* Kill live server and try to fix Windows
changelog_begin
changelog_end
* Undo rm
changelog_begin
changelog_end
* io.grpc:grpc-xxxx to 1.29.0 (from 1.22.1)
io.netty:netty-xxxx to .1.50.Final (from 4.1.37.Final)
io.nett.netty-tcp-native-boringssl-static to 2.0.30.Final (from 2.0.25.Final)
To resolve open vulnerabilities with these versions
netty-4.1.37.Final vulnerabilities
BDSA-2018-4022 (Medium)
BDSA-2019-2610 (Medium)
BDSA-2019-3119 (CVE-2019-16869) (Medium)
BDSA-2020-0130 (Medium)
BDSA BDSA-2019-4230 (CVE-2019-20445) (Low)
BDSA BDSA-2019-4231 (CVE-2019-20444) (Low)
BDSA BDSA-2020-0666 (CVE-2020-11612) (Low)
BDSA BDSA-2019-2642 (Low)
BDSA BDSA-2019-2649 (Low)
BDSA BDSA-2019-2643 (Low)
CHANGELOG_BEGIN
Upgrade io.grpc:grpc-xxxxx and io.netty:netty-xxx version to latest
released to avoid exposure to reported security vulnerabilities in
currently used versions
CHANGELOG_END
–
* Update spray versions to address vulnerabilities
CVE-2018-18853 and CVE-2018-18854
CHANGELOG_BEGIN
Upgrade io.grpc:grpc-xxxxx and io.netty:netty-xxx version to latest
released to avoid exposure to reported security vulnerabilities in
currently used versions
CHANGELOG_END
* do not change io.grpc version since reflection seems to be misbehaving
* Clarify how to bump grpc/netty/protobuf versions
Also "downgrade" netty to 4.1.48, according to
https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty
CHANGELOG_BEGIN
CHANGELOG_END
* Load protobuf deps after haskell deps to avoid loading an older version of rules_cc
* Upgrade protoc and protobuf-java to 3.11.0
* buildifier reformat
* regen unique int after rebase
* remove commented patch
Co-authored-by: Gerolf Seitz <gerolf.seitz@daml.com>
It should not be an issue, but I know of at least 4 people for whom
having envsubst in dev-env causes their local git to get confused and
spit out many lines of warning on each invocation. This also seems to
make git much slower.
Given that we're only using envsubst in this one place, I think it may
be worth replacing.
CHANGELOG_BEGIN
CHANGELOG_END
First version of static verification tool.
The current state of the tool:
- Reads DAR files.
- Partially evaluates the code.
- Generates constraints for the field and choice to be verified.
- Passes the constraints to an SMT solver.
- Some basic tests.
