* Upgrade protobuf-java and scalapb.
CHANGELOG_BEGIN
CHANGELOG_END
* Bazel: Generate ScalaPB classes without `unknownFields`.
This is very annoying because it means that extractors need an extra
`_` for no good reason. It also breaks Circe's automatic derivation.
Including unknown fields is the default behavior in Protocol Buffers for
Java now, and we'll probably have to reckon with it at some point, but
let's kick that can down the road a little.
* Upgrade protobuf-java to 3.17.1.
This is identical to 3.17.0, but let's track it anyway.
* Bazel: Move ScalaPB versions into their own file.
They don't need to go into the generated one.
* client_server_build - user defined outputs
Let the user define a list of output files on client_server_build.
changelog_begin
changelog_end
* daml-script runner expose main(config)
changelog_begin
changelog_end
* Add an example Daml ledger export to the docs
* Build Daml ledger export
changelog_begin
changelog_end
* sh_inline_test
Support toolchain arguments to sh_inline_test. Usefuly for make variable
expansion, e.g. for the POSIX toolchain.
* Add a golden test for Daml ledger export
* Test args files as well
* Use sed from POSIX toolchain
* Add normalization comment to top-level comment
* Ignore trailing CR on Windows
The JSON formatting of the args file uses Windows line endings on
Windows. Therefore, the args.json output technically differs from the
expected output. We're happy to ignore the difference in CRLF here.
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* client_server runner - use temp dir for port file
The sandbox will not overwrite an already existing port file but instead
fail, or worse, silently ignore the error and leave the port-file empty.
changelog_begin
changelog_end
* sandbox: Fail if writing the port file fails
So far this was being silently ignored, leaving a pre-existing port-file
untouched.
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
* check whether collection.compat is unused when compiling for Scala 2.12
- Instead of always suppressing warnings for collection.compat._,
we should only do it for Scala 2.13
- We can also reduce boilerplate by automatically adding this
option when both silencer_plugin and collection-compat are
present
CHANGELOG_BEGIN
CHANGELOG_END
* remove unused import
* remove another unused import
* remove even more unused imports
* missed compat dependency
* more missed compat dependencies
* missed compat dependency
* use scala_deps in scaladoc_jar
- #8423 inlined the major version expansion, but this seems to
have been prior to proper support by scaladoc_jar
* restore custom handling of participant-integration-api
- fixing scaladoc_jar isn't worth it for a single case, as with
deps vs scala_deps
* Add a REPL for each Scala target, for debugging.
For each Scala library target, this adds a `*_repl` target that can be
built and run to provide a Scala REPL with access to the library and all
its dependencies.
To use:
```
$ bazel build //ledger/ledger-on-memory:ledger-on-memory_repl
INFO: Invocation ID: f1c4ec07-68e7-4bc2-a182-40f6ea035110
INFO: Analyzed target //ledger/ledger-on-memory:ledger-on-memory_repl (32 packages loaded, 714 targets configured).
INFO: Found 1 target...
Target //ledger/ledger-on-memory:ledger-on-memory_repl up-to-date:
bazel-bin/ledger/ledger-on-memory/ledger-on-memory_repl.jar
bazel-bin/ledger/ledger-on-memory/ledger-on-memory_repl
INFO: Build completed successfully, 7 total actions
$ ./bazel-bin/ledger/ledger-on-memory/ledger-on-memory_repl
Welcome to Scala 2.12.13 (OpenJDK 64-Bit Server VM, Java 1.8.0_272).
Type in expressions for evaluation. Or try :help.
scala> import scala.concurrent.ExecutionContext.Implicits.global
scala> import com.google.protobuf.ByteString
scala> import com.daml.ledger.on.memory._
scala> import com.daml.ledger.participant.state.kvutils.OffsetBuilder
scala> import com.daml.ledger.participant.state.kvutils.Raw
scala> import com.daml.ledger.participant.state.kvutils.api.LedgerRecord
scala> val state = InMemoryState.empty
state: com.daml.ledger.on.memory.InMemoryState = com.daml.ledger.on.memory.InMemoryState@ff21443
scala> state.write { (log, state) => Future { log += LedgerRecord(OffsetBuilder.fromLong(1), Raw.LogEntryId(ByteString.copyFromUtf8("A")), Raw.Envelope(ByteString.EMPTY)) } }
```
The REPL target has the tag "manual", so will only be built on demand.
CHANGELOG_BEGIN
CHANGELOG_END
* Turns out you can run a REPL with `bazel run`.
* participant-integration-api: Build Oracle tests, but don't run them.
CHANGELOG_BEGIN
CHANGELOG_END
* triggers: Switch to an environment variable for enabling Oracle tests.
* http-json: Switch to an environment variable for enabling Oracle tests.
* Disable running Oracle tests by default, not building them.
* triggers/service: Remove unused test dependencies.
* Switch from `@silent` to `@nowarn`.
This annotation is native to Scala 2.12.13+ and 2.13.2+. It replaces
most usages of `@silent`.
I had to get creative about a couple of use cases that didn't work.
Specifically:
1. Suppressing deprecation warnings works, but Scala 2.12 erroneously
complains that the `@nowarn` is unnecessary. I had to suppress
this warning too with `-Ywarn-unused:-nowarn`.
2. I can't seem to suppress the warning, "The outer reference in this
type test cannot be checked at run time." Instead, I have
refactored the code to remove the warning.
We still need to use the silencer plugin to suppress some warnings about
unused imports (because of compatibility between Scala 2.12 and 2.13),
but this means we no longer need the library, and therefore it is not a
transitive dependency that downstream consumers need to worry about.
CHANGELOG_BEGIN
CHANGELOG_END
* Add some comments around `@nowarn` support.
* language-support/scala: Fix a warning suppression.
* Revert to the default warnings.
Compatibility was complaining.
* compatibility: Use the same Scala version as the root.
It's confusing to get out of sync. `compatibility` was using Scala
2.12.12 while the root was on 2.12.13.
This brings them in sync to 2.12.13.
CHANGELOG_BEGIN
CHANGELOG_END
* Factor out Scala artifacts into scala_version.bzl.
* Bazel: Explicitly specify artifact hashes for Scala 2.13.
Not strictly necessary, as we _happen_ to depend on the same point
release as the Bazel Scala plugin, but this is incidental and we don't
want to bank on it.
* Upgrade Scala 2.12 to v2.12.13.
This is being pulled in anyway because of Maven/Gradle/etc's fun
"favor the most recent" resolution mechanism. The version of Akka we
depend upon transitively depends on Scala 2.12.13, so any downstream
consumers will see that as the Scala version required.
Bringing the Daml repo in line means no more confusion.
CHANGELOG_BEGIN
- [Scala Bindings] If you are using Daml on Scala 2.12, it now depends
on Scala v2.12.13 (from v2.12.12).
CHANGELOG_END
* Scala 2.12.13 is the default version in our pinned version of nixpkgs.
* Upgrade Scala 2.13's Wartremover version.
* Rename `scala_version_rule` to `scala_version_configure`.
* Add a test case to ensure the Scala versions are the same everywhere.
* Add tests for the Scala JAR versions in maven_install_*.json
* gatling-utils: Change the sort order of the expected CSV in tests.
I don't know why this changed, but it seems to be stable.
* compatibility: `scala_version` -> `scala_version_configure`.
* Bazel: Disable the Scala version tests on Windows.
* compatibility: Upgrade Wartremover to Scala 2.12.13.
052f69cde9
broke the structure of our protobufs by changing the import prefix
from "." to an empty string which ends up flattening the whole
file. This PR changes the default for tars to match the old
behavior. We can’t just change the default at the function level since
the prefix is used outside of pkg_tar and in those places an empty
string is required :(
changelog_begin
changelog_end
* Merge Maven uploads for different Scala versions
It turns out Maven will abort an existing staging operation if you
create a new one. This means our jobs race against each other. We
could try to fix that by either sequencing the jobs in a clever
way (annoying and can break things like rerunning if only parts
failed), or by creating more profiles (unclear if you can even have
two profiles for the same group id, even if you do, it’s annoying to
merge).
So in this PR I (grudgingly) merged both uploads into the Haskell
script. This isn’t all bad:
1. It moves some logic from bash embedded in yaml string literals into
Haskell code.
2. It duplicates some versions but it removes duplication in other
places so overall not too much worse.
3. It does however, make things slower. We don’t run this stuff in
parallel. That said, the release step is relatively small (< 5min) and
it only runs on Linux.
We could add CLI arguments to make the Scala versions configurable for
local development. Given that this is blocking releases, I wanted to
get something in that works first and then see what we need in that regard.
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
* .
changelog_begin
changelog_end
Some of our targets, e.g., //ledger/ledger-api-health have no
dependencies. On Scala 2.13, an empty classpath option produces a
bunch of confusing but afaict harmless warnings, on 2.12 it doesn’t
but there is still no point in passing an empty classpath so we can
omit it on both.
changelog_begin
changelog_end
* Replaced diff with diffx
* Explicitly added magnolia and mercator dependencies to fix automatical type class derivation
* CHANGELOG_BEGIN
CHANGELOG_END
* Removed unnecessary Diff type class instance for Seq[T]
* Removed ai.x.diff leftovers
* Added an explanatory comment for magnolia and mercator dependencies
* Formatted changes
* Added optional scaladoc parameter to Bazel's da_scala_library_suite()
* Formatted changes
This fixes Scaladoc and our pom file generation.
It also clears up the confusing error around gatling and removes a
redundant dependency on sbt (no idea why we had that in the first
place) both of which resulted in Scala 2.12 dependencies in our 2.13
lockfile which is obviously bad.
With this, we should now be ready to publish Scala 2.13 artifacts once
the ledger API test tool PR lands.
changelog_begin
changelog_end
* Draw the rest of the Scala 2.13 owl
Not quite but pretty close and this switches us over from inclusions
to exclusions which makes it much easier to track.
Ledger API test tool should be fixed by #8821. Non-repudiation needs a
tiny bit of work since unwrapArray doesn’t work the same on 2.13 but
shouldn’t be hard to fix.
changelog_begin
changelog_end
* Fix ScriptService tests
Those tests were all dumb. They asserted on a fixed order while the
function to sort the things was broken so we ended up with the random
Map order which is unsurprisingly not the same.
This is easily fixed by fixing the sort function.
There is also a second issue with query not sorting.
changelog_begin
changelog_end
* Turns out if you fix one test the next one breaks
And clearly nobody ever tested this or give this a second thought.
changelog_begin
changelog_end
We can’t upgrade to the latest version 1.35.0 since it depends on a
newer version of Bazel but we can at least go to 1.34.1. We also have
to patch absl and grpc itself to fix mingw support and some undeclared
inclusions on Windows. Both patches are taken from upstream PRs (the
mingw one has not been merged yet).
There is also a small patch to grpc-haskell-core. I’ve started working
on upstreaming that but the nix build system is a mess and I haven’t
quite managed to get it working there.
changelog_begin
changelog_end
Disabling it per target works nicely for compilation but it gets
annoying in ghci since the warnings are still triggered. We could
disable it everywhere but I think the warning is generally useful. I
tried patching proto3-suite to use DerivingStrategies but that doesn’t
work because haskell-src is dead and doesn’t support it. So for now
adding it to the per-file list seems like the best option.
changelog_begin
changelog_end
* Clean broken entries from the Bazel cache
This is hopefully a somewhat reasonable workaround for the "output not
created" errors that keep annoying us.
For now, this is just part of the hourly cronjob but we could move it
somewhere else if desired.
changelog_begin
changelog_end
* Fix GCS credentials
changelog_begin
changelog_end
Fixes#8573.
Fixes#8574.
We set the LOCA_ARCHIVE environment variable when a locales archive
exists and the variable is unset.
CHANGELOG_BEGIN
CHANGELOG_END
Fixes#8573.
Fixes#8574.
We ship the `locale-archive` packaged with `nix` along binaries and set
the LOCALE_ARCHIVE environment variable. This makes sure the shipped
binaries behave the same as in our dev-env.
CHANGELOG_BEGIN
CHANGELOG_END
This PR updates scalafmt and enables trailingCommas =
multiple. Unfortunately, scalafmt broke the version field which means
we cannot fully preserve the rest of the config. I’ve made some
attempts to stay reasonably close to the original config but couldn’t
find an exact equivalent in a lot of cases. I don’t feel strongly
about any of the settings so happy to change them to something else.
As announced, this will be merged on Saturday to avoid too many conflicts.
changelog_begin
changelog_end
The one thing that is still missing is making the generated Scala code
from the codegen compatible with Scala 2.13 so the examples are
excluded for now.
changelog_begin
changelog_end
* Port the rest //daml-lf/... to Scala 2.13
Draw the rest of the owl
changelog_begin
changelog_end
* Update daml-lf/encoder/src/main/scala/com/digitalasset/daml/lf/archive/testing/DamlLfEncoder.scala
Co-authored-by: Remy <remy.haemmerle@daml.com>
Co-authored-by: Remy <remy.haemmerle@daml.com>
This only compiles `damlc-bootstrap` as a Haskell binary and `damlc` is
a simple symlink with additional runfiles.
We now need to define `damlc@ghci` manually since it is no longer
defined automatically. The manual definition was tested with
```
$ bazel run //:damlc@ghci --define ghci_data=True
> :main ide -d
```
changelog_begin
changelog_end
Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
As we strive for more inclusiveness, we are becoming less comfortable
with historically-charged terms being used in our everyday work.
This is targeted for merge on Dec 26, _after_ the necessary
corresponding changes at both the GitHub and Azure Pipelines levels.
CHANGELOG_BEGIN
- DAML Connect development is now conducted from the `main` branch,
rather than the `master` one. If you had any dependency on the
digital-asset/daml repository, you will need to update this parameter.
CHANGELOG_END
* Port //daml-lf/data to Scala 2.13
changelog_begin
changelog_end
* factor common ImmArraySeq code to version-agnostic file
- ImmArraySeq itself is agnostic; the 2.12 and 2.13 versions contain
implementation mixins/superclasses for parts that must be specific. The 2.13
version will collapse into the agnostic version when 2.12 support is no longer
desired
* factor common InsertOrdMap code to version-agnostic file
- InsertOrdMap itself is agnostic; the 2.12 and 2.13 versions contain
implementation mixins/superclasses for parts that must be specific. The 2.13
version will collapse into the agnostic version when 2.12 support is no longer
desired
* factor common InsertOrdSet code to version-agnostic file
- InsertOrdSet itself is agnostic; the 2.12 and 2.13 versions contain
implementation mixins/superclasses for parts that must be specific. The 2.13
version will collapse into the agnostic version when 2.12 support is no longer
desired
* factor Map removal
* Move ImmArraySeq back into ImmArray
changelog_begin
changelog_end
* Type assertion instead of symbol
changelog_begin
changelog_end
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Build //libs-scala/... on 2.13
One test is unfortunately disabled at the moment since I utterly
failed to figure out why I get a ClassNotFoundException on 2.13.
changelog_begin
changelog_end
* Copyright headers
changelog_begin
changelog_end
* I can’t bazel today
changelog_begin
changelog_end
* Apply suggestions from code review
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
* Update libs-scala/resources/src/main/2.13/com/daml/resources/UnitCanBuildFrom.scala
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* No split on view
changelog_begin
changelog_end
Co-authored-by: Samir Talwar <samir.talwar@digitalasset.com>
Co-authored-by: Stephen Compall <stephen.compall@daml.com>
* Add a Scala 2.13 build pipeline
This adds initial support for multiple Scala versions controlled via
the DAML_SCALA_VERSION env var and a CI job to make sure we don’t
regress. For now we only test //libs-scala/ports/... which seemed like
the easiest starting point I could find. We can incrementally expand
that over time.
changelog_begin
changelog_end
* Document pinning
changelog_begin
changelog_end
* Address review comments
changelog_begin
changelog_end
This is necessary to at least attempt an upgrade to 2.13 and
generally, I want to keep our rulesets up2date. rules-scala forces the
version of scalatest so we have to bump that at the same time.
This requires changes to basically all Scala test suites since the
import structure has changed and a bunch of things (primarily
scalacheck support) got split out.
Apologies for the giant PR, I don’t see a way to keep it smaller.
changelog_begin
changelog_end
